Allow choosing the julia binary (juliaup compatible) from a notebook

[jkrumbiegel]

Currently, the Malt workers execute julia by copying the first argument from Base.julia_cmd() which is run from the server. This means that in the quarto story, one would be stuck with whatever julia version runs the quarto server. This will be julia by default but can be overridden.

We should add an option to set this directly for a notebook. This should be juliaup compatible, I think, because that's the easiest way to have multiple julias installed on a system. Juliaup uses the syntax julia +1.9.2 which means that it's not a different binary but only an argument that makes the difference here. We should make sure that it still makes sense to pass that as the "binary" because giving binary julia and starting the exeflags with +1.9.2 will break if there's anything between.

We currently have the same problem on the quarto side as deno wants a binary path for QUARTO_JULIA so one cannot pass julia +1.9.2 and such to it either. This could be changed, though, if julia +1.9.2 was given one would just have to split the first part off and pass that as the binary, and the rest as arguments. I guess we could do the same here.

[MichaelHatherly]

Serialization is used to send data across from server to workers. That makes having different julia versions for different workers a slight challenge.

[davidanthoff]

Just as a FYI, I'm still planning to add support for selecting a Julia version via a manifest to Juliaup. So, my general hope is that in the not too distant future, most users will use Juliaup in a way where they just specify a Julia project at startup in some form, and then that automatically determines which Julia version is launched. In that scenario the role of the +X syntax would be more of a niche override scenario.

I don't know how Quarto handles Julia projects, but I think in my ideal world it would just automatically use a project if it finds one either in the same folder as the quarto file, or in a parent folder. And then eventually, once Juliaup adds support for that, this could also automatically select the correct Julia version that is stored in the manifest.

Re the serialization stuff, you could take a look at JSONRPC.jl. We use that for all communication in the VS Code extension, and the scenario here seems very, very similar, and it would solve the different Julia versions problem.

[davidanthoff]

Oh, and I'm seeing that --project=@. is the default already, so that is great!

[MichaelHatherly]

Just as a FYI, I'm still planning to add support for selecting a Julia version via a manifest to Juliaup.

That would definitely be great to have!

Oh, and I'm seeing that --project=@. is the default already, so that is great!

Yes, we settled on that default since it seemed the most sane behaviour.

Re the serialization stuff, you could take a look at JSONRPC.jl. We use that for all communication in the VS Code extension, and the scenario here seems very, very similar, and it would solve the different Julia versions problem.

Will have a look over that, thanks for the pointer.

[jkrumbiegel]

I thought that as long as the server process has no lower version number than the workers, the server should be fine reading the serializations from the workers. Also the types we send over are a limited number of basic types from Base like Strings, Dicts, byte arrays. We can have a look whether JSON RPC would be better than that.

While the juliaup selection through the manifest could be cool, I'm not sure if the mechanism for quarto should be limited to juliaup? At least we should be able to override. Right now we can't choose anything because Malt.jl doesn't support it.