Freeze the _sysconfigdata_* module

[FFY00]

Feature or enhancement

Proposal:

After generating _sysconfigdata_*, we could freeze it, like we do to the rest of the standard library.

This should remove the necessity for workarounds such as having to manually set PYTHONPATH to the _sysconfigdata_* directory when running WASM artifacts:

cpython/Tools/wasm/wasi.py

Lines 226 to 227 in 151d1bf

	"ENV_VAR_NAME": "PYTHONPATH",
	"ENV_VAR_VALUE": f"/{sysconfig_data}",

It should also make it easier to build a fully self-contained statically-linked relocatable interpreter.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

[zooba]

Is it better to freeze it? Or generate native code to extend _sysconfig with the values?

[FFY00]

I would like to keep things the same with -Xfrozen_modules=off. Unless the freezing implementation gets too complex, I think it would be better to keep it as a module.

[zooba]

I'd expect the build order to get complex - I'm pretty sure _sysconfigdata gets generated after CPython is built, but to freeze it you need to generate it halfway through the build. All the variables must be available at that point, so it's doable, but it might be a pain.

Then again, having a .py version of the data is no doubt incredibly helpful for some users, as they can get information about an install without having to run it (which may not even be possible). So generating the .py earlier and using the existing freeze infrastructure may be best for everyone.

Although the frozen modules are there to improve startup time, so they have to balance binary size with likelihood of being imported. I don't know that _sysconfigdata gets loaded every time? If it does, then sure, but if it's only imported on occasion then it might be better to keep it totally separate.

[FFY00]

I'd expect the build order to get complex - I'm pretty sure _sysconfigdata gets generated after CPython is built, but to freeze it you need to generate it halfway through the build. All the variables must be available at that point, so it's doable, but it might be a pain.

It's not much of a pain really, it just depends on Makefile and pyconfig.h, so we can use the bootstrap or --with-build-python interpreter to run the script that generates _sysconfigdata_*.py.

Then again, having a .py version of the data is no doubt incredibly helpful for some users, as they can get information about an install without having to run it (which may not even be possible). So generating the .py earlier and using the existing freeze infrastructure may be best for everyone.

I am not proposing to remove the .py file. Additionally, we are already shipping a JSON file on POSIX (GH-127302), it shouldn't be difficult to also add this to the Windows build.

Although the frozen modules are there to improve startup time, so they have to balance binary size with likelihood of being imported. I don't know that _sysconfigdata gets loaded every time? If it does, then sure, but if it's only imported on occasion then it might be better to keep it totally separate.

Right, that's a good point. It doesn't get loaded every time. Depending on how folks feel about it, we can either use the existing freeze infrastructure, or split it into an extension.

[zooba]

Depending on how folks feel about it, we can either use the existing freeze infrastructure, or split it into an extension.

We already have a _sysconfig extension, so I'd rather see it "split" into there rather than another extension.