feat: publish AppBundler distributions through juliaup - #45
Conversation
The tarball recipe was Linux-only: it hardcoded stage(...; platform =
Linux(arch)) and shipped only a bash launcher + install.sh + .desktop, so
building it on a macOS/Windows runner cross-compiled to Linux and failed
("Cross-compilation to Linux from Darwin is not supported").
- Tarball carries an `os` field; bundle() stages Linux/MacOS/Windows(arch)
accordingly. New --target-os {linux|macos|windows} flag (defaults to host).
- Per-OS files: unix keeps juliaimg_main.sh (made portable: no `readlink -f`,
which BSD/macOS lack) + install.sh (now with a Darwin branch); windows adds
juliaimg_main.bat + install.ps1. The .desktop entry stays Linux-only.
- Packing stays .tar.gz on every OS (Windows 10+ ships tar), reusing TarPack.
This produces a relocatable, UNSIGNED bundle on all three platforms, avoiding
macOS notarization and Windows MSIX code-signing entirely.
Adds the pieces needed to install a bundled Julia distribution with `juliaup add <channel>`, closing the four tasks of issue PeaceFounder#43. Builds on the Tarball target from PeaceFounder#36, which already emits a tree juliaup accepts. juliaup needs no fork for this: it downloads from whatever host JULIAUP_SERVER points at, so a distribution is published as a handful of static files — GitHub Pages is enough — while the tarballs stay on the releases page. - `AppBundler.Juliaup` writes the version database for all 13 client target triples plus the four *DBVERSION pointer files. It mirrors the upstream database by default so stock channels keep working, and resolves the database version above the public one. - `appbundler juliaup <project_dir>` builds that site, and `install_juliaup_workflow()` installs a GitHub Actions workflow that builds the tarballs, attaches them to the release and deploys the database to Pages. - Client wrappers point JULIAUP_SERVER at the distribution and isolate JULIAUP_DEPOT_PATH, leaving a stock juliaup installation untouched. Two details are easy to get wrong by hand and are encoded with tests. juliaup splits build metadata on "." and reads the second component as the architecture, so a build tag containing dots resolves to a nonsense architecture — JuliaHub hit this with early Dyad releases. And juliaup ignores a database whose version is not greater than the one built into its binary, reporting nothing at all when it does. A tarball's canonical name now carries the OS. Without it a release matrix uploads one archive per platform under the same asset name and the database cannot tell them apart. Verified end to end against a real juliaup client over loopback HTTP, for both relative and absolute UrlPath, and by building the CmdApp example and installing it with `juliaup add`. Closes PeaceFounder#43 Assisted-by: AI
|
Thanks, there's a lot of useful stuff in here. I'm not planning to take this PR forward though. The design is already settled and written up in #43, and tarballs are going to come from JuliaImg (#41) rather than AppBundler itself, so a fair bit of this pulls in a different direction. VersionDB is the exception. That part is genuinely useful, and I think it would be better off as its own package anyway — the JuliaUp database is fiddly enough that it deserves somewhere it can be tested on its own, and the tooling for the Pages action could live there too. If you'd be up for splitting it out and registering it, I'd use it, and I'm happy to review. |
|
Ok @JanisErdmanis. I'm working on decoupling VersionDB from this repository in a separate one and let you know.
Any opinion? |
|
Thanks — that's a clear steer, and fair. I had not read #43 and #41 closely enough before building on top of AppBundler, and you're right that the tarball side pulls against where you're taking this. I've split VersionDB out as you suggested: https://github.com/s-celles/JuliaupDistributions.jl It carries the database plus the publishing side you mentioned:
The AppBundler coupling is gone: no preference lookups, no shared parameter machinery, and Mustache replaced by plain interpolation. 160 tests, including Aqua. The one I'd point you at is The two silent failure modes you'd expect to bite are covered by tests: a database version at or below the public one is ignored with no diagnostic, and a build tag containing dots makes the architecture parse as nonsense (the thing JuliaHub hit with early Dyad releases). Both were guesses I only trusted after checking them against the published upstream databases. Not registered yet. One thing worth your opinion before I do: the General naming guidelines say to avoid Closing this one — the useful part now lives where you suggested. Review very welcome. |
|
This looks like it's headed in the right direction. For a full demo, you could show that it's possible to take Julia tarballs from the julialang website and register them in the version database (perhaps under a mocked name). Then use that version database to install Julia via JuliaUp. As I understand it, JuliaUp can also point to a local database, which would be useful for testing before deploying the version database to GitHub Pages. As for the name, it seems it should contain "Database" — perhaps I'm currently unable to pursue this further, as I have deadlines to meet on other AppBundler tasks. The JuliaUp integration and related refactoring will definitely be the first task I start on November 1st, and I expect to complete it by the end of November. |
|
The demo is done: It mocks nothing. It reads the public database, takes the actual Your hint about a local database was the right one, and it is what the demo Not registered yet, and the name may still move — the General guidelines To be straight about the timing: I am not just building this speculatively — Assisted-by: AI |
Closes #43.
Relationship to #36
This branch contains the three commits from #36 unchanged, with @SimonDanisch as their author, because the juliaup work needs a tarball to install. Only the last commit is mine. If #36 lands first this rebases down to that single commit; if you would rather review the juliaup layer alone, I can rebase onto #36 and open this against that branch instead.
What this adds
juliaupdownloads from whatever hostJULIAUP_SERVERpoints at, so distributing a bundled Julia through it needs no fork and no dedicated infrastructure — only a handful of static files, while the tarballs stay on the releases page. This is the approach JuliaHub uses for Dyad.AppBundler.Juliaup— writes the version database for all 13 client target triples plus the four*DBVERSIONpointer files. Mirrors the upstream database by default sorelease,ltsand every stock channel keep working for users pointed at your server;--no-mirrorpublishes your channels alone for an air-gapped site.appbundler juliaup <project_dir>— builds that static site, andAppBundler.install_juliaup_workflow()installs a GitHub Actions workflow that builds a tarball per platform, attaches them to the release, and deploys the database to GitHub Pages.<app>-juliaup/<app>-juliaplus PowerShell equivalents, settingJULIAUP_SERVERand isolatingJULIAUP_DEPOT_PATHso a user's stockjuliaupis untouched, as in the gist linked from the issue.docs/src/juliaup.md), aCHANGELOG.md, ajustfile, andllms.txt/llms-full.txtgenerated with the documentation.Two silent failure modes, encoded with tests
Both cost real time to discover, so they are covered rather than documented alone:
juliaupsplits build metadata on.and reads the second component as the architecture. A build tag containing dots shifts that index and the entry resolves to a nonsense architecture. JuliaHub hit this —1.11.8+dyad-2.1.0-rc3.x64.linux.gnuparses its architecture as1— and later entries readdyad-2x1x0-rc3. AppBundler sanitises the tag.juliaupreplaces its cached database only when the published number exceeds both the number compiled into its binary and its local copy, and logs nothing when it does not. The version is resolved above upstream automatically, and the workflow seeds the previously published number so republishing keeps climbing.Also worth knowing: a database is named after the target triple of the
juliaupclient, not the Julia build. The Linux client is musl-static but runs on glibc hosts, so a Linux build must appear in both the-musland-gnudatabases (upstream publishes them as identical files), and each database carries every architecture its client can execute — including x86_64 in the Apple Silicon database, for Rosetta 2.Changes to the tarball target from #36
<app>-<version>-<os>-<arch>.tar.gz). With--target-os, three runners otherwise upload three different archives under the same asset name and the database cannot tell them apart. This is a behaviour change to add Tarball recipe #36 and the one thing here I would most want a second opinion on.TarPack.packreportedArgumentError: Collection has multiple elementsinstead of its own explanatory error — theonly()threw before the guard ran.--target-bundle=tarballraised aMethodErrorwith thejuliacbundler; the missingJuliaCBundlemethod is added.Verification
test/juliaup.jl(132 assertions, fixture trimmed from the real upstream database, no network) andtest/tarball.jlpass. Docs build warning-free.test/juliaup_e2e.jlruns the whole flow against a realjuliaupclient over a loopback HTTP server —juliauppermits plain HTTP on loopback — covering both a relative and an absoluteUrlPath. Opt in withJULIA_RUN_JULIAUP_E2E=true; it skips whenjuliaupis not onPATH. The absolute case is what makes the Pages-plus-releases layout work, so it is proven rather than assumed.Manually, end to end:
Not verified here: a real GitHub Pages deploy and an HTTPS
juliaup add. The workflow is reviewed by inspection; the loopback test exercises the same code path.test/stage.jlandtest/juliac.jlfail in my environment for the pre-existing reasons CI works around (xvfb-run,Pkg.Apps.add("JuliaC")); withjuliacinstalled,test/integrity.jlpasses.Known limitation
A distribution installed through
juliaupis launched asjulia, not through thebin/<app>launcher the tarball also ships — and that launcher is what setsUSER_DATA. Without it AppEnv falls back to a temporary depot on Linux, so packages a user adds on top of the distribution do not persist. Documented; settingUSER_DATArestores persistence. Happy to address it differently if you would rather the juliaup path set it another way.Assisted-by: AI