script/cibuild-setup-py: install all dev requirements for testing

[z-bsod]

install all requirements stored in requirements-dev instead of only a short hardcoded list for the test of the test after build

[ross]

install all requirements stored in requirements-dev instead of only a short hardcoded list for the test of the test after build

It intentionally doesn't install requirements-dev.txt before running tests to prevent things in that file from being accidental production/runtime dependencies. Those requirements are test-time and rather than development related.

You can see the distinction in setup.py,

octodns-template/setup.py

Lines 23 to 28 in 7011a39

	tests_require = (
	'pytest',
	'pytest-cov',
	'pytest-network',
	# TODO: other test-time requirements
	)

and

octodns-template/setup.py

Lines 34 to 49 in 7011a39

	extras_require={
	'dev': tests_require
	+ (
	# we need to manually/explicitely bump major versions as they're
	# likely to result in formatting changes that should happen in their
	# own PR. This will basically happen yearly
	# https://black.readthedocs.io/en/stable/the_black_code_style/index.html#stability-policy
	'black>=24.3.0,<25.0.0',
	'build>=0.7.0',
	'isort>=5.11.5',
	'pyflakes>=2.2.0',
	'readme_renderer[md]>=26.0',
	'twine>=3.4.2',
	),
	'test': tests_require,
	},

Iirc i looked into installing the test specific requirements with pip in the past with something like

pip install dist/*$VERSION*.whl[test]

but ran into problems/bugs that prevented it from functioning correctly. Even then it would allow things in the test section to be accidental "production" requirements so there's a small(er) risk to that.

[ross]

Oh, forgot to mention, setup.py also explicitly doesn't use requirements.txt and sources everything from setup.py since that's what it's specifically testing. requirements*.txt is used for all the other testing.

[z-bsod]

i was aware of the difference between the requirements for test and dev in setup.py.

If you list the same test requirements in the ci script as in the requirements_tests in the setup.py file there should be no difference regarding accidential requirements to test dependencies but i don't see any way of resolving this as you can't run the tests without the test requirements.

i just tried running the tests with pip install "${wheelfile}[test]" and it seems to work correctly the only issue is that pip doesn't support file globbing when executed this way.

I updated the script in my MR and tried the modified version in the repo of the autodns-provider and it does seem to work correctly there (example run)

[ross]

i just tried running the tests with pip install "${wheelfile}[test]" and it seems to work correctly the only issue is that pip doesn't support file globbing when executed this way.

🆒 It was long enough ago that I don't remember what problems(s) I hit with it. Maybe this was it or it was another since addressed shortcoming with pip.

I updated the script in my MR and tried the modified version in the repo of the autodns-provider and it does seem to work correctly there (example run)

👍