Skip to content

refactor(workflows): validate multiarch images once #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
imbajin merged 9 commits into from
Jul 22, 2026
+502 −398
Merged

refactor(workflows): validate multiarch images once #23

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
dfd4cb3
feat(workflows): add graph publish precheck
imbajin Jul 21, 2026
8ca4883
fix(workflows): guard compose cleanup
imbajin Jul 21, 2026
09c0e58
refactor(workflows): promote tested images
imbajin Jul 22, 2026
2b29e81
perf(workflows): select arm build strategy
imbajin Jul 22, 2026
ab66b10
refactor(workflows): consolidate staged builds
imbajin Jul 22, 2026
3c5e436
refactor(workflows): publish multiarch once
imbajin Jul 22, 2026
85338fb
fix(workflows): drop checkout credentials
imbajin Jul 22, 2026
267dfdb
fix(workflows): harden publish gates
imbajin Jul 22, 2026
80b9922
fix(workflows): retry emulated builds
imbajin Jul 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
271 changes: 263 additions & 8 deletions .github/workflows/_publish_pd_store_server_reusable.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,52 +270,307 @@ jobs:

- name: Start compose stack with local images
run: |
if [ ! -f "docker/docker-compose.yml" ]; then
echo "ERROR: docker/docker-compose.yml not found in $REPOSITORY_URL@$SOURCE_SHA"
echo "Please update the compose file path in this workflow."
if [ -f "docker/docker-compose.dev.yml" ]; then
compose_file="docker/docker-compose.dev.yml"
elif [ -f "docker/docker-compose.yml" ]; then
compose_file="docker/docker-compose.yml"
echo "WARN: docker/docker-compose.dev.yml is unavailable; using legacy docker/docker-compose.yml"
else
echo "ERROR: no supported compose file found in $REPOSITORY_URL@$SOURCE_SHA"
echo "Expected docker/docker-compose.dev.yml or docker/docker-compose.yml."
exit 1
fi
echo "COMPOSE_FILE=$compose_file" >> "$GITHUB_ENV"

cat > /tmp/hg-ci-patch-server-config.sh <<'PATCH_SERVER'
#!/usr/bin/env bash
set -euo pipefail

set_property() {
local file="$1" key="$2" value="$3" escaped
escaped=${key//./\.}
if grep -qE "^[[:space:]]*${escaped}[[:space:]]*=" "$file"; then
sed -E -i "s|^[[:space:]]*${escaped}[[:space:]]*=.*|${key}=${value}|" "$file"
else
printf '%s=%s\n' "$key" "$value" >> "$file"
fi
}

set_yaml_scalar() {
local file="$1" key="$2" value="$3"
if grep -qE "^[[:space:]]*${key}:" "$file"; then
sed -E -i "s|^([[:space:]]*)${key}:.*|\1${key}: ${value}|" "$file"
else
printf '%s: %s\n' "$key" "$value" >> "$file"
fi
}

set_property ./conf/rest-server.properties batch.max_write_threads 4
set_property ./conf/rest-server.properties restserver.min_free_memory 32
set_yaml_scalar ./conf/gremlin-server.yaml threadPoolWorker 4
set_yaml_scalar ./conf/gremlin-server.yaml gremlinPool 4

echo "Applied CI Server resource configuration"
exec ./docker-entrypoint.sh
PATCH_SERVER
chmod 755 /tmp/hg-ci-patch-server-config.sh

cat > /tmp/docker-compose.ci.override.yml <<COMPOSE_OVERRIDE
services:
pd:
image: hg-ci/pd:precheck
container_name: hg-pd
pull_policy: never
mem_limit: 1024m
cpus: 2.0
environment:
JAVA_OPTS: >-
-Xms128m -Xmx128m
-XX:ActiveProcessorCount=2
-XX:MaxMetaspaceSize=192m
-XX:MaxDirectMemorySize=128m
-XX:+UseContainerSupport
-Dlog4j2.asyncLoggerConfigRingBufferSize=8192
-DAsyncLoggerConfig.RingBufferSize=8192
-Dthread.pool.grpc.core=8
-Dthread.pool.grpc.max=32
-Dthread.pool.grpc.queue=512
-Djob.interruptableThreadPool.core=1
-Djob.interruptableThreadPool.max=8
-Djob.interruptableThreadPool.queue=256
-Djob.uninterruptibleThreadPool.core=0
-Djob.uninterruptibleThreadPool.max=4
-Djob.uninterruptibleThreadPool.queue=128
-Dpartition.default-shard-count=1
-Dpartition.store-max-shard-count=12
store:
image: hg-ci/store:precheck
container_name: hg-store
pull_policy: never
mem_limit: 1024m
cpus: 2.0
environment:
JAVA_OPTS: >-
-Xms128m -Xmx128m
-XX:ActiveProcessorCount=2
-XX:MaxMetaspaceSize=192m
-XX:MaxDirectMemorySize=128m
-XX:+UseContainerSupport
-Dlog4j2.asyncLoggerConfigRingBufferSize=8192
-DAsyncLoggerConfig.RingBufferSize=8192
-Drocksdb.total_memory_size=134217728
-Drocksdb.write_buffer_size=2097152
-Drocksdb.min_write_buffer_number_to_merge=2
-Dthread.pool.grpc.core=8
-Dthread.pool.grpc.max=32
-Dthread.pool.grpc.queue=512
-Dthread.pool.scan.core=4
-Dthread.pool.scan.max=16
-Dthread.pool.scan.queue=256
-Djob.interruptableThreadPool.core=2
-Djob.interruptableThreadPool.max=8
-Djob.interruptableThreadPool.queue=256
-Djob.uninterruptibleThreadPool.core=0
-Djob.uninterruptibleThreadPool.max=4
-Djob.uninterruptibleThreadPool.queue=128
-Dquery.push-down.threads=8
-Dquery.push-down.fetch_batch=1000
-Dquery.push-down.fetch_timeout=30000
-Dquery.push-down.memory_limit_count=10000
-Dquery.push-down.index_size_limit_count=10000
-Draft.disruptorBufferSize=512
-Draft.metrics=false
-Draft.maxReplicatorInflightMsgs=32
-Draft.maxEntriesSize=64
-Draft.maxBodySize=262144
server:
image: hg-ci/server:precheck
container_name: hg-server
pull_policy: never
mem_limit: 1536m
cpus: 4.0
entrypoint:
- /usr/bin/dumb-init
- "--"
- /opt/hg-ci/patch-server-config.sh
environment:
JAVA_OPTS: >-
-Xms256m -Xmx384m
-XX:ActiveProcessorCount=4
-XX:MaxMetaspaceSize=256m
-XX:MaxDirectMemorySize=128m
-XX:+UseContainerSupport
-Dlog4j2.asyncLoggerConfigRingBufferSize=8192
-DAsyncLoggerConfig.RingBufferSize=8192
volumes:
- type: bind
source: /tmp/hg-ci-patch-server-config.sh
target: /opt/hg-ci/patch-server-config.sh
read_only: true
COMPOSE_OVERRIDE

docker compose \
-p hg-ci-precheck \
-f docker/docker-compose.yml \
-f "$compose_file" \
-f /tmp/docker-compose.ci.override.yml \
up -d --wait --wait-timeout "$WAIT_TIMEOUT_SEC"

docker compose -p hg-ci-precheck -f docker/docker-compose.yml -f /tmp/docker-compose.ci.override.yml ps
docker compose -p hg-ci-precheck -f "$compose_file" -f /tmp/docker-compose.ci.override.yml ps

- name: Verify CI resource limits
run: |
set -euo pipefail

assert_memory_limit() {
local container="$1" expected_bytes="$2" actual_bytes
actual_bytes="$(docker inspect --format '{{.HostConfig.Memory}}' "$container")"
if [ "$actual_bytes" != "$expected_bytes" ]; then
echo "Unexpected memory limit for ${container}: expected ${expected_bytes}, got ${actual_bytes}"
exit 1
fi
echo "${container} memory limit: ${actual_bytes} bytes"
}

assert_memory_limit hg-pd 1073741824
assert_memory_limit hg-store 1073741824
assert_memory_limit hg-server 1610612736

for container in hg-pd hg-store hg-server; do
docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}' "$container" \
| grep '^JAVA_OPTS='
docker exec "$container" bash -lc "ps -eo args | grep '[j]ava'"
done
docker exec hg-server grep -E \
'^(batch.max_write_threads|restserver.min_free_memory)=' \
/hugegraph-server/conf/rest-server.properties
docker exec hg-server grep -E \
'^[[:space:]]*(threadPoolWorker|gremlinPool):' \
/hugegraph-server/conf/gremlin-server.yaml

docker stats --no-stream hg-pd hg-store hg-server

- name: Verify integration endpoints
run: |
curl -fsS --connect-timeout 3 --max-time 8 "http://127.0.0.1:8620/v1/health" >/dev/null
curl -fsS --connect-timeout 3 --max-time 8 "http://127.0.0.1:8520/v1/health" >/dev/null
curl -fsS --connect-timeout 3 --max-time 8 "http://127.0.0.1:8080/versions" >/dev/null

- name: Import bundled example graph
run: |
set -euo pipefail

docker exec hg-server test -r /hugegraph-server/scripts/example.groovy
docker exec hg-server bash -lc \
'cd /hugegraph-server && bin/gremlin-console.sh -- -e scripts/example.groovy' \
| tee /tmp/hg-ci-example-import.log

grep -Fq '>>>> query all vertices: size=6' /tmp/hg-ci-example-import.log
grep -Fq '>>>> query all edges: size=6' /tmp/hg-ci-example-import.log

- name: Verify bundled graph with Gremlin CRUD
env:
CI_VERTEX_NAME: hg-ci-${{ github.run_id }}-${{ github.run_attempt }}
run: |
set -euo pipefail

gremlin_request() {
local label="$1" script="$2" payload response_file
response_file="/tmp/hg-ci-gremlin-${label}.json"
payload="$(
jq -nc --arg gremlin "$script" \
'{gremlin:$gremlin,bindings:{},language:"gremlin-groovy",aliases:{graph:"DEFAULT-hugegraph",g:"__g_DEFAULT-hugegraph"}}'
)"
curl --fail-with-body -sS \
--compressed \
--connect-timeout 3 \
--max-time 30 \
-H 'Content-Type: application/json' \
--data-binary "$payload" \
http://127.0.0.1:8080/gremlin \
| tee "$response_file"
jq -e '.status.code == 200' "$response_file" >/dev/null
}

baseline_script="[[vertices:g.V().count().next(),edges:g.E().count().next(),names:g.V().values('name').order().toList(),markoCreated:g.V().has('person','name','marko').out('created').values('name').order().toList()]]"
gremlin_request baseline "$baseline_script"
jq -e '
.result.data[0].vertices == 6 and
.result.data[0].edges == 6 and
.result.data[0].names == ["josh","lop","marko","peter","ripple","vadas"] and
.result.data[0].markoCreated == ["lop"]
' /tmp/hg-ci-gremlin-baseline.json >/dev/null

create_script="ci=graph.addVertex(T.label,'person','name','${CI_VERTEX_NAME}','age',40,'city','CI');marko=g.V().has('person','name','marko').next();marko.addEdge('knows',ci,'date','ci','weight',0.1d);graph.tx().commit();'${CI_VERTEX_NAME}'"
gremlin_request create "$create_script"
jq -e --arg name "$CI_VERTEX_NAME" '.result.data == [$name]' \
/tmp/hg-ci-gremlin-create.json >/dev/null

read_created_script="[[vertices:g.V().count().next(),edges:g.E().count().next(),matches:g.V().has('person','name','marko').out('knows').has('name','${CI_VERTEX_NAME}').count().next()]]"
gremlin_request read-created "$read_created_script"
jq -e '
.result.data[0].vertices == 7 and
.result.data[0].edges == 7 and
.result.data[0].matches == 1
' /tmp/hg-ci-gremlin-read-created.json >/dev/null

update_script="g.V().has('person','name','${CI_VERTEX_NAME}').property('age',41).iterate();graph.tx().commit();g.V().has('person','name','${CI_VERTEX_NAME}').values('age').next()"
gremlin_request update "$update_script"
jq -e '.result.data == [41]' /tmp/hg-ci-gremlin-update.json >/dev/null

delete_script="ci=g.V().has('person','name','${CI_VERTEX_NAME}').next();g.V(ci).bothE().drop().iterate();g.V(ci).drop().iterate();graph.tx().commit();[[vertices:g.V().count().next(),edges:g.E().count().next()]]"
gremlin_request delete "$delete_script"
jq -e '
.result.data[0].vertices == 6 and
.result.data[0].edges == 6
' /tmp/hg-ci-gremlin-delete.json >/dev/null

gremlin_request final "$baseline_script"
jq -e '
.result.data[0].vertices == 6 and
.result.data[0].edges == 6 and
.result.data[0].names == ["josh","lop","marko","peter","ripple","vadas"]
' /tmp/hg-ci-gremlin-final.json >/dev/null

docker stats --no-stream hg-pd hg-store hg-server

- name: Dump compose logs on failure
if: ${{ failure() }}
run: |
docker compose -p hg-ci-precheck -f docker/docker-compose.yml -f /tmp/docker-compose.ci.override.yml logs --no-color --tail=200 || true
docker compose -p hg-ci-precheck -f "$COMPOSE_FILE" -f /tmp/docker-compose.ci.override.yml ps || true
docker stats --no-stream hg-pd hg-store hg-server || true
for container in hg-pd hg-store hg-server; do
docker inspect --format '{{json .State.Health}}' "$container" || true
done
for artifact in /tmp/hg-ci-example-import.log /tmp/hg-ci-gremlin-*.json; do
if [ -f "$artifact" ]; then
echo "===== ${artifact} ====="
cat "$artifact"
fi
done
docker compose -p hg-ci-precheck -f "$COMPOSE_FILE" -f /tmp/docker-compose.ci.override.yml logs --no-color --tail=200 || true

- name: Stop compose stack
if: ${{ always() }}
run: |
set -euo pipefail

docker compose \
-p hg-ci-precheck \
-f docker/docker-compose.yml \
-f "$COMPOSE_FILE" \
-f /tmp/docker-compose.ci.override.yml \
down -v --remove-orphans || true
down -v --remove-orphans

leftover_containers="$(docker ps -aq --filter label=com.docker.compose.project=hg-ci-precheck)"
leftover_networks="$(docker network ls -q --filter label=com.docker.compose.project=hg-ci-precheck)"
leftover_volumes="$(docker volume ls -q --filter label=com.docker.compose.project=hg-ci-precheck)"
if [ -n "$leftover_containers$leftover_networks$leftover_volumes" ]; then
echo "Compose cleanup left resources behind"
docker ps -a --filter label=com.docker.compose.project=hg-ci-precheck
docker network ls --filter label=com.docker.compose.project=hg-ci-precheck
docker volume ls --filter label=com.docker.compose.project=hg-ci-precheck
exit 1
fi
echo "Compose cleanup removed all project containers, networks, and volumes"

- name: Post-check cleanup
if: ${{ always() }}
Expand Down
25 changes: 23 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,22 @@ changing public images or production cache state.
`pd/store/server` is the most important publishing flow in this repository and uses a dedicated reusable workflow:
[`.github/workflows/_publish_pd_store_server_reusable.yml`](./.github/workflows/_publish_pd_store_server_reusable.yml).

The strict integration precheck builds local amd64 PD, Store, and HStore Server
images, starts the upstream `docker/docker-compose.dev.yml` topology with
`pull_policy: never`, and runs a functional graph check before any image is
published. Compatible source revisions that have the same service contract but
only contain `docker/docker-compose.yml` use that legacy file as a fallback. The
check executes the Server image's bundled
`/hugegraph-server/scripts/example.groovy` file, verifies the six-vertex,
six-edge sample graph, then performs separate Gremlin read, create, update, and
delete requests. The final query must return to the original 6V/6E baseline.

The precheck override constrains the three JVMs and Store buffers for a small
CI workload. PD and Store are limited to 1 GiB each, Server to 1.5 GiB, while
heap, direct memory, RocksDB, Raft, and worker queues use a low-memory test
profile. These settings are functional-test limits, not production guidance or
a performance baseline.

```text
source branch (master / release-x.y.z)
|
Expand All @@ -110,7 +126,7 @@ changing public images or production cache state.
|
v
integration_precheck (optional)
(compose health check for pd/store/server-hstore)
(low-memory compose + bundled example graph + Gremlin CRUD)
|
v
publish_amd64 (matrix x4 modules)
Expand Down Expand Up @@ -178,12 +194,17 @@ Reusable workflows are the real implementation layer.
`_publish_pd_store_server_reusable.yml` handles the pd/store/server flow:

- shared source SHA resolution and latest hash gate
- strict integration precheck for pd/store/server (hstore backend, `hugegraph/server`)
- strict low-memory integration precheck for pd/store/server (hstore backend, `hugegraph/server`)
- import of the Server image's bundled `example.groovy` graph and Gremlin CRUD validation
- staged image publication with `*-amd64` then `*-arm64`
- manifest merge to final tag (`latest` or release version)
- remove temporary `*-amd64` and `*-arm64` tags after successful manifest publish
- standalone server smoke test for `hugegraph/hugegraph`

The current precheck intentionally uses a 1 PD + 1 Store + 1 Server topology so
it fits standard GitHub-hosted runners. A full 3 PD + 3 Store + 3 Server compose
gate remains a TODO for a larger runner or a reliable lower-resource simulation.

Wrapper workflows provide the source repository, branch, and mode-specific inputs.
Standard wrappers may also pass `build_matrix_json`, while the pd/store/server matrix is defined inside `_publish_pd_store_server_reusable.yml`.

Expand Down