Skip to content

feat: publish AppBundler distributions through juliaup - #45

Closed
s-celles wants to merge 4 commits into
PeaceFounder:mainfrom
s-celles:feat/juliaup-distribution
Closed

feat: publish AppBundler distributions through juliaup#45
s-celles wants to merge 4 commits into
PeaceFounder:mainfrom
s-celles:feat/juliaup-distribution

Conversation

@s-celles

@s-celles s-celles commented Sep 5, 2026

Copy link
Copy Markdown

Closes #43.

Relationship to #36

This branch contains the three commits from #36 unchanged, with @SimonDanisch as their author, because the juliaup work needs a tarball to install. Only the last commit is mine. If #36 lands first this rebases down to that single commit; if you would rather review the juliaup layer alone, I can rebase onto #36 and open this against that branch instead.

What this adds

juliaup downloads from whatever host JULIAUP_SERVER points at, so distributing a bundled Julia through it needs no fork and no dedicated infrastructure — only a handful of static files, while the tarballs stay on the releases page. This is the approach JuliaHub uses for Dyad.

  • AppBundler.Juliaup — writes the version database for all 13 client target triples plus the four *DBVERSION pointer files. Mirrors the upstream database by default so release, lts and every stock channel keep working for users pointed at your server; --no-mirror publishes your channels alone for an air-gapped site.
  • appbundler juliaup <project_dir> — builds that static site, and AppBundler.install_juliaup_workflow() installs a GitHub Actions workflow that builds a tarball per platform, attaches them to the release, and deploys the database to GitHub Pages.
  • Client wrappers<app>-juliaup / <app>-julia plus PowerShell equivalents, setting JULIAUP_SERVER and isolating JULIAUP_DEPOT_PATH so a user's stock juliaup is untouched, as in the gist linked from the issue.
  • Docs (docs/src/juliaup.md), a CHANGELOG.md, a justfile, and llms.txt / llms-full.txt generated with the documentation.

Two silent failure modes, encoded with tests

Both cost real time to discover, so they are covered rather than documented alone:

  • juliaup splits build metadata on . and reads the second component as the architecture. A build tag containing dots shifts that index and the entry resolves to a nonsense architecture. JuliaHub hit this — 1.11.8+dyad-2.1.0-rc3.x64.linux.gnu parses its architecture as 1 — and later entries read dyad-2x1x0-rc3. AppBundler sanitises the tag.
  • juliaup replaces its cached database only when the published number exceeds both the number compiled into its binary and its local copy, and logs nothing when it does not. The version is resolved above upstream automatically, and the workflow seeds the previously published number so republishing keeps climbing.

Also worth knowing: a database is named after the target triple of the juliaup client, not the Julia build. The Linux client is musl-static but runs on glibc hosts, so a Linux build must appear in both the -musl and -gnu databases (upstream publishes them as identical files), and each database carries every architecture its client can execute — including x86_64 in the Apple Silicon database, for Rosetta 2.

Changes to the tarball target from #36

  • A tarball's canonical name now carries the OS (<app>-<version>-<os>-<arch>.tar.gz). With --target-os, three runners otherwise upload three different archives under the same asset name and the database cannot tell them apart. This is a behaviour change to add Tarball recipe #36 and the one thing here I would most want a second opinion on.
  • TarPack.pack reported ArgumentError: Collection has multiple elements instead of its own explanatory error — the only() threw before the guard ran.
  • --target-bundle=tarball raised a MethodError with the juliac bundler; the missing JuliaCBundle method is added.

Verification

test/juliaup.jl (132 assertions, fixture trimmed from the real upstream database, no network) and test/tarball.jl pass. Docs build warning-free.

test/juliaup_e2e.jl runs the whole flow against a real juliaup client over a loopback HTTP server — juliaup permits plain HTTP on loopback — covering both a relative and an absolute UrlPath. Opt in with JULIA_RUN_JULIAUP_E2E=true; it skips when juliaup is not on PATH. The absolute case is what makes the Pages-plus-releases layout work, so it is proven rather than assumed.

Manually, end to end:

appbundler build examples/CmdApp --target-bundle=tarball   # 197 MB tarball, single root, bin/julia
appbundler juliaup examples/CmdApp --build-dir=site ...    # 13 databases, 4 pointers, wrappers
juliaup add cmdapp-0.1.0                                   # installs
julia +cmdapp-0.1.0 -e 'import CmdApp'                     # the bundled package loads

Not verified here: a real GitHub Pages deploy and an HTTPS juliaup add. The workflow is reviewed by inspection; the loopback test exercises the same code path. test/stage.jl and test/juliac.jl fail in my environment for the pre-existing reasons CI works around (xvfb-run, Pkg.Apps.add("JuliaC")); with juliac installed, test/integrity.jl passes.

Known limitation

A distribution installed through juliaup is launched as julia, not through the bin/<app> launcher the tarball also ships — and that launcher is what sets USER_DATA. Without it AppEnv falls back to a temporary depot on Linux, so packages a user adds on top of the distribution do not persist. Documented; setting USER_DATA restores persistence. Happy to address it differently if you would rather the juliaup path set it another way.

Assisted-by: AI

SimonDanisch and others added 4 commits September 5, 2026 09:35
The tarball recipe was Linux-only: it hardcoded stage(...; platform =
Linux(arch)) and shipped only a bash launcher + install.sh + .desktop, so
building it on a macOS/Windows runner cross-compiled to Linux and failed
("Cross-compilation to Linux from Darwin is not supported").

- Tarball carries an `os` field; bundle() stages Linux/MacOS/Windows(arch)
  accordingly. New --target-os {linux|macos|windows} flag (defaults to host).
- Per-OS files: unix keeps juliaimg_main.sh (made portable: no `readlink -f`,
  which BSD/macOS lack) + install.sh (now with a Darwin branch); windows adds
  juliaimg_main.bat + install.ps1. The .desktop entry stays Linux-only.
- Packing stays .tar.gz on every OS (Windows 10+ ships tar), reusing TarPack.

This produces a relocatable, UNSIGNED bundle on all three platforms, avoiding
macOS notarization and Windows MSIX code-signing entirely.
Adds the pieces needed to install a bundled Julia distribution with
`juliaup add <channel>`, closing the four tasks of issue PeaceFounder#43. Builds on the
Tarball target from PeaceFounder#36, which already emits a tree juliaup accepts.

juliaup needs no fork for this: it downloads from whatever host JULIAUP_SERVER
points at, so a distribution is published as a handful of static files — GitHub
Pages is enough — while the tarballs stay on the releases page.

- `AppBundler.Juliaup` writes the version database for all 13 client target
  triples plus the four *DBVERSION pointer files. It mirrors the upstream
  database by default so stock channels keep working, and resolves the database
  version above the public one.
- `appbundler juliaup <project_dir>` builds that site, and
  `install_juliaup_workflow()` installs a GitHub Actions workflow that builds
  the tarballs, attaches them to the release and deploys the database to Pages.
- Client wrappers point JULIAUP_SERVER at the distribution and isolate
  JULIAUP_DEPOT_PATH, leaving a stock juliaup installation untouched.

Two details are easy to get wrong by hand and are encoded with tests. juliaup
splits build metadata on "." and reads the second component as the
architecture, so a build tag containing dots resolves to a nonsense
architecture — JuliaHub hit this with early Dyad releases. And juliaup ignores a
database whose version is not greater than the one built into its binary,
reporting nothing at all when it does.

A tarball's canonical name now carries the OS. Without it a release matrix
uploads one archive per platform under the same asset name and the database
cannot tell them apart.

Verified end to end against a real juliaup client over loopback HTTP, for both
relative and absolute UrlPath, and by building the CmdApp example and
installing it with `juliaup add`.

Closes PeaceFounder#43

Assisted-by: AI
@JanisErdmanis

Copy link
Copy Markdown
Member

Thanks, there's a lot of useful stuff in here.

I'm not planning to take this PR forward though. The design is already settled and written up in #43, and tarballs are going to come from JuliaImg (#41) rather than AppBundler itself, so a fair bit of this pulls in a different direction.

VersionDB is the exception. That part is genuinely useful, and I think it would be better off as its own package anyway — the JuliaUp database is fiddly enough that it deserves somewhere it can be tested on its own, and the tooling for the Pages action could live there too. If you'd be up for splitting it out and registering it, I'd use it, and I'm happy to review.

@s-celles

s-celles commented Sep 5, 2026

Copy link
Copy Markdown
Author

Ok @JanisErdmanis. I'm working on decoupling VersionDB from this repository in a separate one and let you know.
But I'm not sure what naming may be more appropriate

  • JuliaupDistributions.jl
  • DistributionChannels.jl
  • VersionChannels.jl

Any opinion?

@s-celles

s-celles commented Sep 5, 2026

Copy link
Copy Markdown
Author

Thanks — that's a clear steer, and fair. I had not read #43 and #41 closely enough before building on top of AppBundler, and you're right that the tarball side pulls against where you're taking this.

I've split VersionDB out as you suggested: https://github.com/s-celles/JuliaupDistributions.jl

It carries the database plus the publishing side you mentioned:

  • the 13 client target triples with their compatible-architecture expansion, the four *DBVERSION pointer files, version-string construction, merging, and the database-version bump
  • publish, writing the complete static tree a client reads, with mirroring so stock channels keep working
  • client wrappers setting JULIAUP_SERVER and isolating JULIAUP_DEPOT_PATH
  • a GitHub Actions template for the Pages deploy

The AppBundler coupling is gone: no preference lookups, no shared parameter machinery, and Mustache replaced by plain interpolation. JSON is the only non-stdlib dependency, so depending on it costs you almost nothing.

160 tests, including Aqua. The one I'd point you at is test/e2e_tests.jl: it stands up a loopback HTTP server, publishes to it, and installs the channel with a real juliaup clientjuliaup allows plain HTTP on loopback, so the whole flow is testable offline. It covers both a relative and an absolute UrlPath, the latter being what makes the Pages-plus-releases layout work.

The two silent failure modes you'd expect to bite are covered by tests: a database version at or below the public one is ignored with no diagnostic, and a build tag containing dots makes the architecture parse as nonsense (the thing JuliaHub hit with early Dyad releases). Both were guesses I only trusted after checking them against the published upstream databases.

Not registered yet. One thing worth your opinion before I do: the General naming guidelines say to avoid Julia in a package name, and I'd rather hear that from someone who has been through registration than find out in the PR. Happy to rename if you think it'll get pushback.

Closing this one — the useful part now lives where you suggested. Review very welcome.

@s-celles s-celles closed this Sep 5, 2026
@JanisErdmanis

Copy link
Copy Markdown
Member

This looks like it's headed in the right direction. For a full demo, you could show that it's possible to take Julia tarballs from the julialang website and register them in the version database (perhaps under a mocked name). Then use that version database to install Julia via JuliaUp. As I understand it, JuliaUp can also point to a local database, which would be useful for testing before deploying the version database to GitHub Pages.

As for the name, it seems it should contain "Database" — perhaps JuliaUpDatabase.

I'm currently unable to pursue this further, as I have deadlines to meet on other AppBundler tasks. The JuliaUp integration and related refactoring will definitely be the first task I start on November 1st, and I expect to complete it by the end of November.

@s-celles

s-celles commented Sep 5, 2026

Copy link
Copy Markdown
Author

The demo is done: examples/real_julia_demo.jl in
https://github.com/s-celles/JuliaupDistributions.jl

It mocks nothing. It reads the public database, takes the actual release
entry, republishes it under a channel name of its own, and lets juliaup
download and install the real Julia from julialang.org:

Installing Julia 1.12.7+0.x64.linux.gnu
Add Installed Julia channel 'demo-julia'
Julia 1.12.7 installed as the demo-julia channel

Your hint about a local database was the right one, and it is what the demo
uses: juliaup reads its database straight out of the depot, so dropping a
file into $JULIAUP_DEPOT_PATH/juliaup/versiondb-<target>.json is enough — no
server, no Pages deploy. That also exercises the absolute UrlPath path for
free, since an entry republished away from julialang.org can no longer be
relative. Both are now documented, and the demo runs in CI alongside the
loopback end-to-end test.

Not registered yet, and the name may still move — the General guidelines
discourage a Julia prefix, so I would rather settle that with the registry
reviewers than pick something twice. Whatever it ends up as, the API is what
you see now.

To be straight about the timing: I am not just building this speculatively —
I have a desktop application of my own that I want to ship this way, so the
package will keep moving regardless. That is on my side though, not a nudge on
yours; November is entirely fine for the AppBundler end.

Assisted-by: AI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add JuliaUp support for AppBundler-produced distributions

3 participants