Bug: Can't disable dependency that's not on PYPI

[rvanlaar]

Pre-submission checks

• I am not filing an auditing error (false positive or negative). These must be reported to pypa/advisory-database instead.
• I agree to follow the PSF Code of Conduct.
• I have looked through the open issues for a duplicate report.

Expected behavior

I expected pip-audit --strict to ignore dependencies that aren't on Pypi.

Actual behavior

Instead pip-audit --strict gave the following error:

ERROR:pip_audit._cli:django-private-media: Dependency not found on PyPI and could not be audited: django-private-media (0.1.4)
Exit code 1.

Reproduction steps

1. $ virtualenv venv
2. $ source venv/bin/activate
3. $ pip install git+https://github.com/kiesopmaat/django-private-media.git
4. $ pip install pip-audit
5. $ pip-audit --strict

Logs

Additional context

Seems to be related to #157

OS name, version, and architecture

Ubuntu 24.04.1 LTS on AMD64

pip-audit version

pip-audit 2.7.3

pip version

pip 24.0

Python version

Python 3.12.3

[di]

I think this is a duplicate of #599?

[woodruffw]

Yep, I think so -- @rvanlaar does that package come from a private index? If so, does passing --extra-index-url allow it to resolve?

(NB that this won't find vulnerabilities for that package, since it's not on PyPI. But it will prevent the resolution error.)

[rvanlaar]

It looks like it's related to #599 but it's slightly different. We install directly from github:
pip install git+https://github.com/kiesopmaat/django-private-media.git

So there's no index url.

(venv) roland@MiX ~/wb60> pip freeze > requirements.txt
(venv) roland@MiX ~/wb60> pip-audit --strict --extra-index-url=https://github.com/kiesopmaat/django-private-media.git -r requirements.txt 
ERROR:pip_audit._cli:django-private-media: Dependency not found on PyPI and could not be audited: django-private-media (0.1.4)
(venv) roland@MiX ~/wb60 [1]> pip-audit --strict --extra-index-url=https://github.com/kiesopmaat/django-private-media -r requirements.txt
ERROR:pip_audit._cli:django-private-media: Dependency not found on PyPI and could not be audited: django-private-media (0.1.4)
(venv) roland@MiX ~/wb60 [1]> 

(NB: I do understand that this won't find vulnerabilities for this package)

[woodruffw]

Gotcha, hmm. I'm actually a little surprised we error in that case; I was pretty sure we only warn when we can't resolve a package on PyPI.

Out of curiosity: does the command complete successfully when you drop --strict? That flag may be telling pip-audit to fail when it can't resolve django-private-media, which isn't what you want in your case (since you're explicitly installing it from a non-index source).

Longer term, I suppose we could handle this more gracefully by seeing where the package was installed from -- IIRC one of the newer packaging PEPs exposes that information, at least in some cases.

[rvanlaar]

I've tried using pip-audit without strict and that passes. When I added a package (xhtml2pdf 0.2.16) with a known vulnerability it fails, i.e. exit code 1.

In my tests I used pip-audit in a virtualenv without specifying a requirements.txt.
Could you elaborate on what strict means, regarding to the dependency collection in our case?

  -S, --strict          fail the entire audit if dependency collection fails
                        on any dependency (default: False)

[woodruffw]

Could you elaborate on what strict means, regarding to the dependency collection in our case?

In your case, all --strict does is cause failures if a package doesn't exist on PyPI. It used to do more things back when pip-audit had more of its own dependency resolution logic, but these days it's only used to turn a skipped dependency into a fatal error.

(The reason this exists is because some users don't always know which dependencies are or aren't on PyPI, so they need a strict mode to fail conservatively and sort things out.)

You can see the context where --strict is checked here:

pip-audit/pip_audit/_cli.py

Lines 510 to 514 in bd25e9f

	for spec, vulns in auditor.audit(source):
	if spec.is_skipped():
	spec = cast(SkippedDependency, spec)
	if args.strict:
	_fatal(f"{spec.name}: {spec.skip_reason}")

TL;DR: Removing --strict should have no impact besides the desired one (not failing on a git+... dependency) in your case.

[rvanlaar]

I appreciate the quickness and depths of the responses. From my opinion this bug can be closed because it was a misunderstanding about what --strict does. I'll leave that up to you.

My suggestion would be to reword the description of --strict to be clearer what it does.

[woodruffw]

I appreciate the quickness and depths of the responses. From my opinion this bug can be closed because it was a misunderstanding about what --strict does. I'll leave that up to you.

Thanks, we appreciate the detail in your responses as well! I agree this is a misunderstanding about what --strict does, and I'll look into making description/doc improvements to clarify this.

Thanks again for your report! Closing.