Bug: generating report on windows fails when contains chinese symbols

[alysenko-nn]

Pre-submission checks

• I am not filing an auditing error (false positive or negative). These must be reported to pypa/advisory-database instead.
• I agree to follow the PSF Code of Conduct.
• I have looked through the open issues for a duplicate report.

Expected behavior

Pip audit generates a report successfully.

Actual behavior

Report fails when the vulnerability contains Chinese symbols

Reproduction steps

Windows 10 Home
run: pip-audit -r reqs.txt

reqs.txt contains:

tensorflow==2.10.0

GHSA-cqvq-fvhr-v6hc
In positions 54306-54308, this line 刘力源

Logs

[2025-07-18T17:22:58.899Z] Traceback (most recent call last): 
[2025-07-18T17:22:58.899Z]   File "C:\Users\Demo\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main  
[2025-07-18T17:22:58.899Z]   return _run_code(code, main_globals, None,  
[2025-07-18T17:22:58.899Z]   File "C:\Users\Demo\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code  
[2025-07-18T17:22:58.899Z]   exec(code, run_globals)  
[2025-07-18T17:22:58.899Z]   File "C:\Users\Demo\AppData\Local\Temp\brainconverter-nightly-venv\Scripts\pip-audit.exe\__main__.py", line 7, in <module>  
[2025-07-18T17:22:58.899Z]   File "C:\Users\Demo\AppData\Local\Temp\brainconverter-nightly-venv\lib\site-packages\pip_audit\_cli.py", line 620, in audit  
[2025-07-18T17:22:58.899Z]   print(formatter.format(result, fixes), file=io)  
[2025-07-18T17:22:58.899Z]   File   "C:\Users\Demo\AppData\Local\Programs\Python\Python39\lib\encodings\cp1252.py", line 19, in encode  
[2025-07-18T17:22:58.899Z]   return codecs.charmap_encode(input,self.errors,encoding_table)[0]  
[2025-07-18T17:22:58.899Z] UnicodeEncodeError: 'charmap' codec can't encode characters in position 54306-  54308: character maps to <undefined>

Additional context

It could be fixed by change cp1252 to utf8 in windows: set PYTHONUTF8=1

OS name, version, and architecture

Windows 10 home

pip-audit version

2.9.0

pip version

25.0.1

Python version

3.9

[woodruffw]

Hi @alysenko-nn, thanks for the report.

I'll be able to triage more in a day or so, but on first look I'm not sure that this is a pip-audit bug -- Windows's choice of an ancient default codepage/text encoding causes all kinds of problems with any program that (reasonably IMO) assumes that user inputs are UTF-8.

We might be able to patch for this as a convenience matter, but I think you're right that set PYTHONUTF8=1 is ultimately the most correct way to handle this: realistically, there's a lot of transitive string API usage in pip-audit, so even if we fix the proximate error in this case it'll likely crop up elsewhere.

See di/pip-api#239 for a related case (where a more general fix was possible, since pip was being spawned in a controlled manner).

[jgerityneurala]

I'm not sure that this is a pip-audit bug -- Windows's choice of an ancient default codepage/text encoding causes all kinds of problems with any program that (reasonably IMO) assumes that user inputs are UTF-8.

I guess it depends on what the goals are for pip-audit when it comes to operability on Windows, i.e. whose feet this reported behavior ends up at. But agreed that the flaw here originates with Windows. This class of problem will probably become less subtle when PEP 686 lands and becomes the norm (so, probably not until 2030 😆).

We might be able to patch for this as a convenience matter

If the goal is that pip-audit "just works" on Windows, I think such a patch would be a good idea, probably to add an explicit encoding in the relevant open(). Shame that there's no way to opt into UTF-8 mode after runtime start-up. For sure the bug we encountered here is addressed by PYTHONUTF8=1, so we have a workaround on our end.

Thanks @woodruffw

[alysenko-nn]

Hi @alysenko-nn, thanks for the report.

I'll be able to triage more in a day or so, but on first look I'm not sure that this is a pip-audit bug -- Windows's choice of an ancient default codepage/text encoding causes all kinds of problems with any program that (reasonably IMO) assumes that user inputs are UTF-8.

We might be able to patch for this as a convenience matter, but I think you're right that set PYTHONUTF8=1 is ultimately the most correct way to handle this: realistically, there's a lot of transitive string API usage in pip-audit, so even if we fix the proximate error in this case it'll likely crop up elsewhere.

See di/pip-api#239 for a related case (where a more general fix was possible, since pip was being spawned in a controlled manner).

@woodruffw Thanks for clarifying

[woodruffw]

Thanks both -- yeah, after consideration I think this is worth fixing on pip-audit's side. I'll have a patch up in a day or two.