Livebook reproducibility regarding installed packages not in a lockfile #1660
-
|
How does Livebook ensure reproducibility regarding package management? Since Elixir Just curious and trying to understand, no real problem right now. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 2 replies
-
|
Hey @linusdm! The We indeed don't use lockfiles for reproducability and rely on the version requirements in the dependency list. So technically when open a notebook from somewhere you may get a slightly different versions than originally, but we expect packages to follow semver, so this should be fine. You can mitigate some of it by specifying exact versions for each package, though there are still sub-dependencies that may be resolved to newer versions. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I see how that mitigates the problem. I guess it's not such a big problem with the Elixir ecosystem being more mindful with the semver conventions. I heard José mention how Livebook tackles the reproducibility issue (here for example, but also in this video), and was curious if I had missed something, or if there was something in the pipeline to lock down dependencies even more. I could see how the lock information could even be embedded in the livebook itself (even the lockfile is semi-readable text information). But it would make the markdown less manageable and a bit verbose. But it could be opt-in for people really wanting to lock down everything very strictly in the notebook itself. Another option is to keep all dependencies in a mix project, and depend on that sole project with As always, thanks for taking the time Jonatan! I'm actually preparing a presentation for FOSDEM next week, which is why I'm having all these detailed questions. I'll try to pay it forward by getting some people interested in trying out Livebook... |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, I think embedding the lockfiles in Markdown, even as a comment is a little bit too much, unless we ever run into actual issues. Good luck with the presentation, sounds great! :) |
Beta Was this translation helpful? Give feedback.
Hey @linusdm!
The
:lockfileand:config_pathoptions are specifically to mirror a setup likeiex -S mixin Livebook (while also allowing notebook-specific deps).We indeed don't use lockfiles for reproducability and rely on the version requirements in the dependency list. So technically when open a notebook from somewhere you may get a slightly different versions than originally, but we expect packages to follow semver, so this should be fine. You can mitigate some of it by specifying exact versions for each package, though there are still sub-dependencies that may be resolved to newer versions.