Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,8 +302,8 @@ Some vendors publish a release feed of their own that is faster and more precise
than the AUR packaging of it. Those packages are `source: local` — Omarchy owns
the PKGBUILD — and declare where releases come from in one of two ways.

A vendor shipping tagged GitHub releases with a checksum manifest asset is pure
data, declared as `upstream` in `.omarchy/package.json` with no code at all:
A vendor shipping tagged GitHub releases is pure data, declared as `upstream`
in `.omarchy/package.json` with no code at all:

```json
"upstream": {
Expand All @@ -316,6 +316,12 @@ data, declared as `upstream` in `.omarchy/package.json` with no code at all:
}
```

`checksums` names the manifest asset the vendor publishes. A vendor publishing
none sets `"digests": true` instead, and the checksums come from the SHA-256
digest GitHub's release API reports for every asset — see
`pkgbuilds/schist-bin/.omarchy/package.json`. Either way the artifacts
themselves are never downloaded.

`{tag}` and `{pkgver}` interpolate into asset names; a leading `v` on the tag is
stripped for `pkgver`; drafts and prereleases are ignored. Only the 100 most
recent releases are considered. The provider fails closed on anything it cannot
Expand Down Expand Up @@ -626,7 +632,7 @@ Minimal examples:
Fields:

- `source`: `aur` or `local`. A `local` package can still follow an upstream release, either declaratively via `upstream` or with an `.omarchy/upstream.sh` hook.
- `upstream`: optional for `local` packages whose vendor ships tagged GitHub releases with a checksum manifest asset. `{ "github": "owner/repo", "checksums": "SHASUMS256.txt", "assets": { "<arch>": "name-{tag}.tar.xz" } }` — see [Sync Upstream Releases](#sync-upstream-releases). Mutually exclusive with `.omarchy/upstream.sh`.
- `upstream`: optional for `local` packages whose vendor ships tagged GitHub releases. `{ "github": "owner/repo", "checksums": "SHASUMS256.txt", "assets": { "<arch>": "name-{tag}.tar.xz" } }`, or `"digests": true` in place of `checksums` to use the release API's per-asset digests — see [Sync Upstream Releases](#sync-upstream-releases). Mutually exclusive with `.omarchy/upstream.sh`.
- `min_release_age`: optional quarantine for upstream releases (`"24h"`, `"2d"`, or bare seconds). The newest release older than the window ships; anything younger waits, and a release whose age cannot be proven fails the sync. Bypass deliberately with `BYPASS_MIN_RELEASE_AGE=1 bin/sync-upstream <package>`.
- `sync`: optional for AUR packages; defaults to `true`. Set `false` for AUR-origin packages that Omarchy maintains manually.
- `aur`: optional AUR package name when it differs from the local package directory, usually for split packages.
Expand Down
99 changes: 95 additions & 4 deletions bin/sync-upstream
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ Usage: $0 [PACKAGE...]
Update packages that track an upstream vendor release feed instead of the AUR.

Packages opt in declaratively through "upstream" in .omarchy/package.json.
Providers cover GitHub Releases with checksum manifests, semver-shaped git
tags whose source URLs can be hashed, and npm dist-tags. See README.md for the
Providers cover GitHub Releases with checksum manifests or API asset digests,
semver-shaped git tags whose source URLs can be hashed, and npm dist-tags. See README.md for the
schemas. Anything outside those conventions may provide
pkgbuilds/<package>/.omarchy/upstream.sh, a hook that reports JSON on stdout:

Expand Down Expand Up @@ -466,8 +466,8 @@ sync_package() {
# paths. Covers release selection (fallback past quarantined releases,
# draft/prerelease filtering, bypass, unchanged version), failure paths
# (unusable tags/timestamps, missing checksums), checksum template mapping
# for both architectures, the min_release_age backstop, the duration parser,
# and manifest validation.
# for both architectures, release API digests, the min_release_age backstop,
# the duration parser, and manifest validation.
cmd_self_test() {
local failures=0

Expand Down Expand Up @@ -567,6 +567,76 @@ EOF
rc=0; github_upstream_release "$pkg" 86400 >/dev/null 2>&1 || rc=$?
check "missing aarch64 checksum fails the sync" "1" "$rc"

# A vendor publishing no manifest: checksums come from the digests the
# release API reports per asset, with nothing fetched beyond the feed.
echo "Release API digests:"
local digpkg="$TEMP_DIR/selftest-digests"
mkdir -p "$digpkg/.omarchy"
printf 'pkgver=1.0.0\npkgrel=1\n' > "$digpkg/PKGBUILD"
cat > "$digpkg/.omarchy/package.json" <<'EOF'
{
"source": "local",
"upstream": {
"github": "example/tool",
"digests": true,
"assets": {
"x86_64": "tool-{pkgver}-1-x86_64.pkg.tar.zst",
"aarch64": "tool-{pkgver}-1-aarch64.pkg.tar.zst"
}
}
}
EOF
FIXTURE_RELEASES=$(jq -n --arg old "$old2d" --arg x "$sum_x19" --arg a "$sum_a19" '[
{tag_name: "v1.9.0", published_at: $old, draft: false, prerelease: false, assets: [
{name: "tool-1.9.0-1-x86_64.pkg.tar.zst", digest: ("sha256:" + $x)},
{name: "tool-1.9.0-1-aarch64.pkg.tar.zst", digest: ("sha256:" + $a)},
{name: "tool-1.9.0-1-x86_64.rpm", digest: "sha256:0000000000000000000000000000000000000000000000000000000000000000"}
]}
]')
FIXTURE_CHECKSUMS="manifest must not be consulted"
out=$(github_upstream_release "$digpkg" 0 2>/dev/null) || out="<error>"
check "x86_64 checksum via the asset digest" "$sum_x19" "$(jq -r '.sha256sums.x86_64[0] // "<none>"' <<<"$out")"
check "aarch64 checksum via the asset digest" "$sum_a19" "$(jq -r '.sha256sums.aarch64[0] // "<none>"' <<<"$out")"

FIXTURE_RELEASES=$(jq -n --arg old "$old2d" --arg x "$sum_x19" '[
{tag_name: "v1.9.0", published_at: $old, draft: false, prerelease: false, assets: [
{name: "tool-1.9.0-1-x86_64.pkg.tar.zst", digest: ("sha256:" + $x)},
{name: "tool-1.9.0-1-aarch64.pkg.tar.zst"}
]}
]')
rc=0; github_upstream_release "$digpkg" 0 >/dev/null 2>&1 || rc=$?
check "asset without a digest fails the sync" "1" "$rc"

FIXTURE_RELEASES=$(jq -n --arg old "$old2d" --arg x "$sum_x19" '[
{tag_name: "v1.9.0", published_at: $old, draft: false, prerelease: false, assets: [
{name: "tool-1.9.0-1-x86_64.pkg.tar.zst", digest: ("sha256:" + $x)},
{name: "tool-1.9.0-2-aarch64.pkg.tar.zst", digest: ("sha256:" + $x)}
]}
]')
rc=0; github_upstream_release "$digpkg" 0 >/dev/null 2>&1 || rc=$?
check "asset re-cut under another release number fails the sync" "1" "$rc"

FIXTURE_RELEASES=$(jq -n --arg old "$old2d" --arg x "$sum_x19" --arg a "$sum_a19" '[
{tag_name: "v1.9.0", published_at: $old, draft: false, prerelease: false, assets: [
{name: "tool-1.9.0-1-x86_64.pkg.tar.zst", digest: $x},
{name: "tool-1.9.0-1-aarch64.pkg.tar.zst", digest: ("sha256:" + $a)}
]}
]')
rc=0; github_upstream_release "$digpkg" 0 >/dev/null 2>&1 || rc=$?
check "digest without the sha256: prefix fails the sync" "1" "$rc"

# The provider enforces the declaration shape itself: scheduled runs reach
# it without validate_package_metadata.
jq '.upstream.checksums = "SHASUMS256.txt"' "$digpkg/.omarchy/package.json" > "$digpkg/both.json"
cp "$digpkg/.omarchy/package.json" "$digpkg/good.json"
cp "$digpkg/both.json" "$digpkg/.omarchy/package.json"
rc=0; github_upstream_release "$digpkg" 0 >/dev/null 2>&1 || rc=$?
check "provider rejects checksums and digests together" "1" "$rc"
jq '.upstream.digests = "true"' "$digpkg/good.json" > "$digpkg/.omarchy/package.json"
rc=0; github_upstream_release "$digpkg" 0 >/dev/null 2>&1 || rc=$?
check "provider rejects a non-boolean digests" "1" "$rc"
cp "$digpkg/good.json" "$digpkg/.omarchy/package.json"

echo "Quarantine backstop:"
local rel st
rel=$(jq -n --arg p "$old2d" '{pkgver: "1.9.0", published_at: $p, sha256sums: {}}')
Expand Down Expand Up @@ -619,6 +689,27 @@ EOF
echo '{"source": "local", "upstream": {"github": "example/tool"}}' > "$agepkg/.omarchy/package.json"
vst=0; validate_package_metadata "$agepkg" >/dev/null || vst=$?
check "upstream without checksums/assets is rejected" "1" "$vst"
echo '{"source": "local", "upstream": {"github": "example/tool", "checksums": "SHASUMS256.txt", "digests": true, "assets": {"x86_64": "a"}}}' > "$agepkg/.omarchy/package.json"
vst=0; validate_package_metadata "$agepkg" >/dev/null || vst=$?
check "upstream with both checksums and digests is rejected" "1" "$vst"
echo '{"source": "local", "upstream": {"github": "example/tool", "digests": "yes", "assets": {"x86_64": "a"}}}' > "$agepkg/.omarchy/package.json"
vst=0; validate_package_metadata "$agepkg" >/dev/null || vst=$?
check "non-boolean digests is rejected" "1" "$vst"
echo '{"source": "local", "upstream": {"github": "example/tool", "checksums": false, "digests": true, "assets": {"x86_64": "a"}}}' > "$agepkg/.omarchy/package.json"
vst=0; validate_package_metadata "$agepkg" >/dev/null || vst=$?
check "checksums: false alongside digests is rejected" "1" "$vst"
echo '{"source": "local", "upstream": {"github": "example/tool", "checksums": "SUMS", "digests": null, "assets": {"x86_64": "a"}}}' > "$agepkg/.omarchy/package.json"
vst=0; validate_package_metadata "$agepkg" >/dev/null || vst=$?
check "digests: null is rejected" "1" "$vst"
echo '{"source": "local", "upstream": {"github": "example/tool", "checksums": "SUMS", "digests": false, "assets": {"x86_64": "a"}}}' > "$agepkg/.omarchy/package.json"
vst=0; validate_package_metadata "$agepkg" >/dev/null || vst=$?
check "digests: false beside a checksums manifest is accepted" "0" "$vst"
echo '{"source": "local", "upstream": {"github": "example/tool", "digests": false, "assets": {"x86_64": "a"}}}' > "$agepkg/.omarchy/package.json"
vst=0; validate_package_metadata "$agepkg" >/dev/null || vst=$?
check "digests: false alone is rejected" "1" "$vst"
cp "$digpkg/.omarchy/package.json" "$agepkg/.omarchy/package.json"
vst=0; validate_package_metadata "$agepkg" >/dev/null || vst=$?
check "the digests declaration shape is accepted" "0" "$vst"
echo '{"source":"local","upstream":{"github":"example/tool","git_tags":"https://example/tool.git","checksums":"sums","assets":{"any":"tool"}}}' > "$agepkg/.omarchy/package.json"
vst=0; validate_package_metadata "$agepkg" >/dev/null || vst=$?
check "multiple provider types are rejected" "1" "$vst"
Expand Down
5 changes: 4 additions & 1 deletion helpers/package-metadata.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
# { "source": "local", "channels": ["edge", "rc", "stable"] }
# { "source": "local", "min_release_age": "24h" }
# { "source": "local", "upstream": { "github": "owner/repo", "checksums": "SHASUMS256.txt", "assets": { "x86_64": "name-{tag}-x64.tar.xz" } } }
# { "source": "local", "upstream": { "github": "owner/repo", "digests": true, "assets": { "x86_64": "name-{tag}-x64.tar.xz" } } }
# { "source": "local", "upstream": { "git_tags": "https://example/repo.git", "tag_pattern": "v{pkgver}", "sources": { "any": ["https://example/archive/{tag}.tar.gz"] } } }
# { "source": "local", "upstream": { "npm": "@scope/package", "sources": { "any": ["{npm_tarball}"] } } }
#
Expand Down Expand Up @@ -459,7 +460,9 @@ validate_package_metadata() {
([has("github"), has("git_tags"), has("npm")] | map(select(.)) | length) == 1
and if has("github") then
(.github | type == "string" and test("\\A[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+\\z"))
and (.checksums | type == "string" and length > 0)
and (if has("checksums") then (.checksums | type == "string" and length > 0) else true end)
and (if has("digests") then (.digests | type == "boolean") else true end)
and (has("checksums") != (has("digests") and .digests == true))
and (.assets | type == "object" and length > 0 and (to_entries | all(
(.key | test("\\A[a-z0-9_]+\\z")) and (.value | type == "string" and length > 0)
)))
Expand Down
59 changes: 43 additions & 16 deletions helpers/upstream-github.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# Declarative upstream providers for bin/sync-upstream.
#
# A package whose upstream ships tagged GitHub releases with a checksum
# manifest asset needs no upstream.sh hook: the whole feed is data, declared
# in .omarchy/package.json --
# A package whose upstream ships tagged GitHub releases needs no upstream.sh
# hook: the whole feed is data, declared in .omarchy/package.json --
#
# "upstream": {
# "github": "jdx/mise",
Expand All @@ -13,12 +12,16 @@
# }
# }
#
# "checksums" names the vendor's manifest asset. A vendor publishing none can
# set "digests": true instead, which reads the SHA-256 digest GitHub's release
# API reports for every asset, so the sync never downloads the artifacts.
#
# {tag} and {pkgver} interpolate into asset names; tags may carry a leading
# "v", which is stripped for pkgver. Drafts and prereleases are ignored. The
# provider emits the same JSON contract as an upstream.sh hook, so
# bin/sync-upstream's validation and min_release_age backstop apply
# unchanged. Git-tag and npm providers below cover projects without a release
# checksum manifest; a feed that fits no convention keeps a bespoke hook.
# unchanged. Git-tag and npm providers below cover projects without GitHub
# releases; a feed that fits no convention keeps a bespoke hook.

# Return the single declarative provider selected by a package. An empty
# result means either no provider or an invalid/ambiguous declaration; the
Expand Down Expand Up @@ -203,17 +206,28 @@ npm_upstream_release() {
# not silently choose from.
github_upstream_release() {
local package_dir="$1" min_age="${2:-0}"
local metadata repo checksums_name
local metadata repo checksums_name use_digests
metadata=$(metadata_file_for_dir "$package_dir")

repo=$(jq -r '(.upstream? | objects | .github) // ""' "$metadata")
if [[ ! "$repo" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]]; then
echo "invalid upstream.github repository: '${repo:-<empty>}'" >&2
return 1
fi
checksums_name=$(jq -r '(.upstream? | objects | .checksums) // ""' "$metadata")
if [[ -z "$checksums_name" ]]; then
echo "upstream.checksums names the checksum manifest asset and is required" >&2
# Enforced here as well as in validate_package_metadata: the scheduled sync
# reaches this provider without running the validator first.
checksums_name=$(jq -r '(.upstream? | objects | .checksums) | strings' "$metadata")
use_digests=$(jq -r '(.upstream? | objects | .digests) | if . == null then "false" elif type == "boolean" then tostring else "invalid" end' "$metadata")
if [[ "$use_digests" == "invalid" ]]; then
echo "upstream.digests must be true or false" >&2
return 1
fi
if [[ -n "$checksums_name" && "$use_digests" == "true" ]]; then
echo "upstream sets both checksums and digests; keep exactly one" >&2
return 1
fi
if [[ -z "$checksums_name" && "$use_digests" != "true" ]]; then
echo "upstream needs either checksums (a manifest asset name) or digests: true" >&2
return 1
fi
local arches
Expand Down Expand Up @@ -281,15 +295,16 @@ github_upstream_release() {
return 0
fi

local checksums
if ! checksums=$(github_fetch_checksums "$repo" "$best_tag" "$checksums_name"); then
local checksums=""
if [[ "$use_digests" != "true" ]] \
&& ! checksums=$(github_fetch_checksums "$repo" "$best_tag" "$checksums_name"); then
echo "could not fetch $checksums_name for $repo $best_tag" >&2
return 1
fi

local jq_args=(--arg pkgver "$best_pkgver" --arg published_at "$best_published_at")
local jq_filter='{pkgver: $pkgver, published_at: $published_at, sha256sums: {}}'
local arch template filename checksum
local arch template filename checksum checksum_source
for arch in "${arches[@]}"; do
if [[ ! "$arch" =~ ^[a-z0-9_]+$ ]]; then
echo "invalid architecture key in upstream.assets: '$arch'" >&2
Expand All @@ -298,11 +313,23 @@ github_upstream_release() {
template=$(jq -r --arg arch "$arch" '.upstream.assets[$arch]' "$metadata")
filename=${template//\{pkgver\}/$best_pkgver}
filename=${filename//\{tag\}/$best_tag}
# Manifest lines are "<sha256> <name>", with the name sometimes prefixed
# "./" (sha256sum of a local path) or "*" (binary-mode marker).
checksum=$(awk -v f="$filename" '$2 == f || $2 == "./" f || $2 == "*" f { print $1; exit }' <<<"$checksums")
if [[ "$use_digests" == "true" ]]; then
# Only a "sha256:<hex>" digest is stripped to its hex; any other shape
# falls through empty and fails the check below.
checksum=$(jq -r --arg tag "$best_tag" --arg name "$filename" '
first(.[] | select(.tag_name == $tag)) | (.assets // [])[]
| select(.name == $name) | (.digest // "")
| if type == "string" and test("\\Asha256:[0-9a-f]{64}\\z") then ltrimstr("sha256:") else "" end
' <<<"$releases")
checksum_source="the release API digest"
else
# Manifest lines are "<sha256> <name>", with the name sometimes prefixed
# "./" (sha256sum of a local path) or "*" (binary-mode marker).
checksum=$(awk -v f="$filename" '$2 == f || $2 == "./" f || $2 == "*" f { print $1; exit }' <<<"$checksums")
checksum_source="$checksums_name"
fi
if [[ ! "$checksum" =~ ^[0-9a-f]{64}$ ]]; then
echo "no valid checksum for $filename in $repo $best_tag $checksums_name" >&2
echo "no valid checksum for $filename in $repo $best_tag $checksum_source" >&2
return 1
fi
jq_args+=(--arg "sum_$arch" "$checksum")
Expand Down
13 changes: 13 additions & 0 deletions pkgbuilds/schist-bin/.omarchy/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"source": "local",
"release_ring": "fast",
"min_release_age": "24h",
"upstream": {
"github": "Infrawrench/schist",
"digests": true,
"assets": {
"x86_64": "schist-{pkgver}-1-x86_64.pkg.tar.zst",
"aarch64": "schist-{pkgver}-1-aarch64.pkg.tar.zst"
}
}
}
44 changes: 44 additions & 0 deletions pkgbuilds/schist-bin/PKGBUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Maintainer: Infrawrench LLC <astrid@infrawrench.com>
pkgname=schist-bin
pkgver=0.11.0
pkgrel=1
# Upstream's own package release, embedded in the asset name. It is
# packages.sh's "release=" and only moves when the packaging changes under
# a version that has already shipped; pkgrel above is this package's own.
# The asset templates in .omarchy/package.json carry the same number.
_relver=1
pkgdesc="Layered image editor with PSD and Affinity support (binary release)"
arch=(x86_64 aarch64)
url="https://github.com/Infrawrench/schist"
license=(MIT)
# The list the payload's own .PKGINFO carries: fontconfig/wayland/
# vulkan-icd-loader are dlopen'd, so namcap flags them "may not be needed"
# — they are.
depends=(fontconfig freetype2 hicolor-icon-theme libxcb libxkbcommon
libxkbcommon-x11 vulkan-icd-loader wayland)
# A Vulkan ICD is needed to draw, but as a hard dependency makepkg -s would
# resolve it to nvidia-utils in the build container; Omarchy installs the right
# driver per machine. libheif is dlopen'd; everything else opens without it.
optdepends=('vulkan-driver: GPU rendering (any Vulkan ICD, vulkan-swrast in software)'
'libheif: HEIC import')
provides=(schist)
conflicts=(schist)
# The released binary ships byte-exact: it is already stripped by the
# release workflow, and its build id is what Sentry symbolication matches.
options=(!strip !debug)
# The release asset is already a pacman-format payload, assembled by
# packaging/linux/packages.sh from the CI build — this just re-wraps its
# usr/ tree, so the dependency lists here have to stay in step with that
# script.
source_x86_64=("$url/releases/download/v$pkgver/schist-$pkgver-$_relver-x86_64.pkg.tar.zst")
source_aarch64=("$url/releases/download/v$pkgver/schist-$pkgver-$_relver-aarch64.pkg.tar.zst")
sha256sums_x86_64=('0fb0a13e579a9bd74880bf49dd69690cc6a5f830db3423f1e448fef516c1bc5f')
sha256sums_aarch64=('1dedd50295ac251206dcc271ce59a918258975a7e87facde2dcee1b03fef265b')

package() {
# makepkg has already extracted the payload into srcdir; its .PKGINFO
# and .MTREE are upstream's and stay behind, only usr/ is packaged.
cp -a "$srcdir/usr" "$pkgdir/"
# The payload keeps the licence under the upstream pkgname.
mv "$pkgdir/usr/share/licenses/schist" "$pkgdir/usr/share/licenses/$pkgname"
}