-
-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Better reproducibility wrapping Python ArviZ #187
Comments
Apparently PythonCall allows for providing a TOML file for dependencies to ensure reproducibility, so that's an alternative. I'm not certain how well PyPlot will work without PyPlot.jl, which uses PyCall not PythonCall. |
PythonPlot is the PythonCall version of PyPlot: https://github.com/stevengj/PythonPlot.jl I've never used it, so I can't comment on the differences. |
I spent some time replacing PyCall with PythonCall, but it turns out to require quite a lot of surgery. PythonCall by default does no conversions between Python and Julia, and we extensively rely on PyCall's automated conversions, so it would require quite some code for ArviZ to "just work" with PythonCall the way it does with PyCall. Plus, every time one installs ArviZ, a new Conda environment is created. Already-downloaded conda packages are not reused, so package installation becomes much slower and internet-bound. I've opted for #234 to hopefully resolve our major dependency issues, and the time that would go toward using PythonCall will instead go toward removing Python as a dependency. |
With #290, we no longer use Python |
One of the ways the Julia ecosystem achieves reproducibility even with binary dependencies is by defining glue packages that wrap the binaries and are tagged for specific versions. e.g. https://github.com/JuliaBinaryWrappers/wget_jll.jl. This way Julia packages can specify exact supported versions for these dependencies.
I'm considering doing something like that here until #130 is completely resolved. Specifically, we would create an
ArviZ_py.jl
(orPyArviZ
) package that only depends onPyCall
. v0.12.0 of the package would install v0.12.0 of the ArviZ Python package upon loading, and upon start-up it would check that the exact Python ArviZ version is installed and otherwise prompt the user to update (what we currently do in this package). When a new Python ArviZ release is made, a new release is tagged there, and then we only update our compat here when we fully support that version (and drop compat for versions we don't support). As an added bonus, we maybe could ensure that on the main branch, the main branch of Python ArviZ is always installed, so testing Julia ArviZ against future releases of Python ArviZ would be as easy as] add ArviZ, ArviZ_py#main
Since Python ArviZ releases are relatively rare, this would not be much of an additional maintenance burden, and it would let us be at least more reproducible than we currently are.
The text was updated successfully, but these errors were encountered: