Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allowing Worker to spawn a different julia #75

Open
MichaelHatherly opened this issue Apr 15, 2024 · 2 comments
Open

Allowing Worker to spawn a different julia #75

MichaelHatherly opened this issue Apr 15, 2024 · 2 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@MichaelHatherly
Copy link
Contributor

Currently Worker doesn't allow for setting the actual julia used, it always uses Base.julia_cmd().

Malt.jl/src/Malt.jl

Lines 103 to 105 in 37e4c4a

function Worker(; env=String[], exeflags=[])
# Spawn process
cmd = _get_worker_cmd(; env, exeflags)

Malt.jl/src/Malt.jl

Lines 254 to 256 in 37e4c4a

function _get_worker_cmd(exe=Base.julia_cmd()[1]; env, exeflags)
return addenv(`$exe --startup-file=no $exeflags $(joinpath(src_path, "worker.jl"))`, String["OPENBLAS_NUM_THREADS=1", Base.byteenv(env)...])
end

Would exposing that in the Worker constructor be something the maintainers would be happy to support? It appears to be needed to be able to support PumasAI/QuartoNotebookRunner.jl#42, which is something we'd like to add to the quarto runner.


Allowing for different julias does of course then run into the issue of serializing data between different versions not necessarily being deserializable by another julia. So that might warrant a separate issue to discuss being able to pick a different serialization method rather than using Serialization.serialize. I can open one for that if the issue of supporting julia selection for Worker is considered worth pursuing itself.

@pankgeorg
Copy link
Member

pankgeorg commented Apr 15, 2024

Sitting next to @fonsp, Fons says great let's do it 😅

(proof: image)

The custom serializer could also be a good idea

@MichaelHatherly
Copy link
Contributor Author

Thanks, I'll see whether I can get a PR sorted sometime this week.

@fonsp fonsp added good first issue Good for newcomers enhancement New feature or request labels Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

3 participants