-
Notifications
You must be signed in to change notification settings - Fork 543
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(toolchain): support freethreaded toolchains #2372
Conversation
Seems like this PR is working to build XLA with Python free-threading using the following command: bazel build \
--@rules_python//python/config_settings:py_freethreaded="yes" \
--define=use_enabled_free_threading=true \
--test_output=all --spawn_strategy=sandboxed \
--override_repository=rules_python=/rules_python \
//xla/... It downloads however two versions of python:
I assume that free-threaded version is used for building python extensions... |
@vfdev-5, I am not fully sure about why you have the toolchain in your cache. Could you please check it with |
In my tests I deleted completely the cache and run xla build from the very beginning. In the console I saw it downloading python for platform |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done with the self-review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So if I remember the toolchain generation logic, then all one has to do is set the flag, and it'll trigger downloading and usage of the free-threaded interpreter?
Otherwise just some small comments, LGTM
@@ -67,19 +69,23 @@ def define_hermetic_runtime_toolchain_impl( | |||
exclude = [ | |||
# Unused shared libraries. `python` executable and the `:libpython` target | |||
# depend on `libpython{python_version}.so.1.0`. | |||
"lib/libpython{major}.{minor}.so".format(**version_dict), | |||
"lib/libpython{major}.{minor}*.so".format(**version_dict), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just checking/curious: I'm guessing the libX.Y*
pattern works because the extracted standalone interpreters only contain appropriately relevant .so files?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is for exclusion, that is why libX.Y*.so
is just as good as libX.Y.so
.
Before this PR freethreaded toolchains were not possible to be used,
this adds the minimum plumbing to get the things working. Coverage
support is also added.
Whilst at it:
Tested:
Closes #2129.
Work towards #2386.