Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new "python-no-pyc" test #11896

Merged
merged 1 commit into from
Feb 19, 2022
Merged

Conversation

tianon
Copy link
Member

@tianon tianon commented Feb 18, 2022

This test validates that the Python images do not contain any pre-compiled .pyc or .pyo files in /usr/local (or /opt, for good measure).

Ideally this would also include all of /usr (not just /usr/local) but then it would also have to filter out results that are from Debian packages (python's default variants are FROM buildpack-deps which includes a fair amount of Python packages).

See docker-library/python#207

(The goal is to simplify how we avoid .pyc and .pyo files in the images but with a safety net to validate that we don't regress.)

@tianon
Copy link
Member Author

tianon commented Feb 18, 2022

$ bashbrew list --uniq --arch-filter python pypy | xargs ./test/run.sh -t python-no-pyc
testing python:3.11.0a5-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.11.0a5-slim-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.11.0a5-buster
	'python-no-pyc' [1/1]...passed
testing python:3.11.0a5-slim-buster
	'python-no-pyc' [1/1]...passed
testing python:3.11.0a5-alpine3.15
	'python-no-pyc' [1/1]...passed
testing python:3.11.0a5-alpine3.14
	'python-no-pyc' [1/1]...passed
testing python:3.10.2-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.10.2-slim-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.10.2-buster
	'python-no-pyc' [1/1]...passed
testing python:3.10.2-slim-buster
	'python-no-pyc' [1/1]...passed
testing python:3.10.2-alpine3.15
	'python-no-pyc' [1/1]...passed
testing python:3.10.2-alpine3.14
	'python-no-pyc' [1/1]...passed
testing python:3.9.10-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.9.10-slim-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.9.10-buster
	'python-no-pyc' [1/1]...passed
testing python:3.9.10-slim-buster
	'python-no-pyc' [1/1]...passed
testing python:3.9.10-alpine3.15
	'python-no-pyc' [1/1]...passed
testing python:3.9.10-alpine3.14
	'python-no-pyc' [1/1]...passed
testing python:3.8.12-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.8.12-slim-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.8.12-buster
	'python-no-pyc' [1/1]...passed
testing python:3.8.12-slim-buster
	'python-no-pyc' [1/1]...passed
testing python:3.8.12-alpine3.15
	'python-no-pyc' [1/1]...passed
testing python:3.8.12-alpine3.14
	'python-no-pyc' [1/1]...passed
testing python:3.7.12-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.7.12-slim-bullseye
	'python-no-pyc' [1/1]...passed
testing python:3.7.12-buster
	'python-no-pyc' [1/1]...passed
testing python:3.7.12-slim-buster
	'python-no-pyc' [1/1]...passed
testing python:3.7.12-alpine3.15
	'python-no-pyc' [1/1]...passed
testing python:3.7.12-alpine3.14
	'python-no-pyc' [1/1]...passed
testing pypy:3.8-7.3.7-bullseye
	'python-no-pyc' [1/1]...passed
testing pypy:3.8-7.3.7-slim
	'python-no-pyc' [1/1]...passed
testing pypy:3.8-7.3.7-buster
	'python-no-pyc' [1/1]...passed
testing pypy:3.8-7.3.7-slim-buster
	'python-no-pyc' [1/1]...passed
testing pypy:3.7-7.3.7-bullseye
	'python-no-pyc' [1/1]...passed
testing pypy:3.7-7.3.7-slim
	'python-no-pyc' [1/1]...passed
testing pypy:3.7-7.3.7-buster
	'python-no-pyc' [1/1]...passed
testing pypy:3.7-7.3.7-slim-buster
	'python-no-pyc' [1/1]...passed
testing pypy:2.7-7.3.6-bullseye
	'python-no-pyc' [1/1]...passed
testing pypy:2.7-7.3.6-slim
	'python-no-pyc' [1/1]...passed
testing pypy:2.7-7.3.6-buster
	'python-no-pyc' [1/1]...passed
testing pypy:2.7-7.3.6-slim-buster
	'python-no-pyc' [1/1]...passed

@@ -0,0 +1 @@
rem TODO implement this check on Windows images too 🙈
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote this because it's a little bit complicated, but on second thought I think we should probably change the message to say it's explicitly not a goal of the Windows images.

This test validates that the Python images do not contain any pre-compiled `.pyc` or `.pyo` files in `/usr/local` (or `/opt`, for good measure).

Ideally this would also include all of `/usr` (not just `/usr/local`) but then it would also have to filter out results that are from Debian packages (`python`'s default variants are `FROM buildpack-deps` which includes a fair amount of Python packages).
@imomaliev
Copy link

Hi, could you please help me understand this part

rem N/A: Windows-based Python images may or may not contain ".pyc" and/or ".pyo" files because they are not a significant contributor to the overall image size

I never used Windows's docker images and see that they are quite large, I assume the main contributor to their size is the base Windows image. After a bit of research, I see that there is also mcr.microsoft.com/windows/nanoserver image and using it as a base in possible future images could require fighting for every byte as in Linux based images. Is similar test checking py[co] for Windows would be hard to implement/maintain or this would be just premature optimization and is not worth the effort?

@tianon
Copy link
Member Author

tianon commented Feb 22, 2022

Yeah, the main size contributor there is definitely the base, and I think the .py[co] files come with the base install of pre-built artifacts from Python. Even if we were to implement docker-library/python#169, I'm not convinced it would make enough of a difference to be worth the hassle.

The flip side of that is that implementing a check for that in pure batch scripting is not going to be a very fun time. 🙈

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants