Python fails to initialize with Conda environments

[fjames003]

This was caused by the changes to use Py_InitializeFromConfig in "ConfgigPEP587" (which is misspelled). This change was made as part of commit 6afe081

Upon trying to initialize Python with a version of Python within a Conda environment (3.10.5 in this case), the following error occurs: "failed to get the Python codec of the filesystem encoding" which is contained in the err_msg of Status return value of Py_InitializeFromConfig.

I have verified that if I modify "ConfgigPEP587" to call Py_Initialize, and the config was setup the way it was previously, by using Python Flags (such as Py_DebugFlag, Py_VerboseFlag, etc) and using Py_SetPythonHome to set PYTHONHOME (the way it was done previously) then Python from within a Conda environment loads without issue.

This is very hard to debug since the Config manipulation was purposely made opaque by using a "Filler" array with a bunch of TConfigOffsets.
For all I know, the offset for TConfigFields.home is wrong, but I am not sure how to verify the provided offset values.

Since the only difference in PythonVersions.pas for Python from within Conda is to call SetPythonHome on the PythonEngine, my working hypothesis is that this issue is related to not properly setting up PYTHONHOME for the config, but it may in fact be some other issue related to the config.

I also see that PyConfig_SetArgv is being imported but never used. I would think that SetProgramArgs should be calling PyConfig_SetArgv, but again, all of this is really unclear because of how opaque the implementation is.

Any ideas for how to handle this issue?

[pyscripter]

The issue may well be related to config offsets.
Could you please try the Embarcadero fork? It uses a similar approach but calculates the offsets. If that works then it is definitely a config offset issue.

In any case I was considering scraping all config 587 stuff and going back to the earlier approach with the flags for versions up to 3.13.

[fjames003]

Thank you for the response.

Your hunch was correct. The Embarcadero Fork loads Python living in a Conda environment without issue.

I also like the approach from Embarcadero to actually define a record that represents PyConfig. That being said, PyConfig.Patch is still a pretty complicated method that could easily have bugs, but at least the resulting structure of the record can more easily be inspected, and of course you get the benefit of using fields to access the various parts of PyConfig.

Is there a reason your approach and Embarcadero's diverge when it comes to initialization?

[pyscripter]

Is there a reason your approach and Embarcadero's diverge when it comes to initialization?

It is a long story. The Embarcadero approach came after mine, but I did not like either. Too hard to maintain. The config structure has been changing in every version of python. With PEP 741 implemented in python 3.14 there is a much better route going forward, and I was thinking of scrapping the PEP 587 stuff and going back to the flags, which are supported up to python 3.13.

[pyscripter]

@fjames003
Fixed. I took the opportunity to check all other offsets...
Could you please check again.