ninja uses more jobs than specified by makeflags #38950
Labels
Comments
2 tasks
|
FWIW this is:
Judging by the comments on that PR, it may materialize in the next few years. |
4 tasks
vbraun
pushed a commit
to vbraun/sage
that referenced
this issue
Feb 18, 2025
`MAKEFLAGS` is an alternative to `MAKE`. The note about `ninja` can stay there until sagemath#38950 is fixed to warn the user e.g. to not run too many other things in parallel. It would be rather surprising if you pass `make -j3` there and have 7 processes running in parallel (which might eat up a lot of RAM and possibly have the OOM killer kill other processes along the way). ### 📝 Checklist <!-- Put an `x` in all the boxes that apply. --> - [x] The title is concise and informative. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [x] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies <!-- List all open PRs that this PR logically depends on. For example, --> <!-- - sagemath#12345: short description why this is a dependency --> <!-- - sagemath#34567: ... --> URL: sagemath#39128 Reported by: user202729 Reviewer(s): Julian Rüth, user202729
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently, if SageMath is built, it will use up more than number of parallel jobs specified by
MAKEFLAGS, becausemakeandninjahas no communication.I notice that there are already tools to parse number of threads (
./src/bin/sage-num-threads.py). At the very least we can throw-jNto ninja, so it would use at most2Njobs.Partial work:
A process tree might look like the following.
For reference, at the moment (I don't fully understand how it work)
pyproject_hooksis used to invokebuild_wheel, which in turn delegates tomesonpy.We need to modify
config_settings, which are done by modifyinghook_inputinIf I understood the code correctly, it suffices to set
config_settings={"compile-args": ["-jN"]}(for some integerN), which is done by settinghook_inputto{"kwargs": {"config_settings": {"compile_args": ["-jN"]}}}.I don't know how
hook_inputis controlled.Looks like it comes from
sdh_build_wheelwhich callspython3 -m build, and you can somehow pass--config-setting=--compile-args=-jNto it using$#build 1.2.2.post1which is in turn managed by
sage/build/pkgs/cython/spkg-install.inIn top level
pyproject.tomlthere isbuild-backend = 'mesonpy'specification. So passing-jNto that backend should be safe?In
sage-env, it already setsSAGE_NUM_THREADSandSAGE_NUM_THREADS_PARALLELenvironment variables.The text was updated successfully, but these errors were encountered: