feat: add Codex Hubble development environment#746
Conversation
dosubot
Bot
added
size:XL
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## master #746 +/- ##
=============================================
- Coverage 62.49% 32.87% -29.63%
+ Complexity 1903 1562 -341
=============================================
Files 262 264 +2
Lines 9541 14310 +4769
Branches 886 1743 +857
=============================================
- Hits 5963 4704 -1259
- Misses 3190 9074 +5884
- Partials 388 532 +144 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
dosubot
Bot
added
size:XXL
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The new environment has clean-worktree startup, lifecycle ownership, image refresh, storage persistence, and local exposure issues that should be fixed before merge. Evidence: six independent review lanes; bash syntax, ShellCheck, environment tests, Compose expansion/dry-run, and Docker image metadata.
| classpath="${RUNTIME_DIR}/hubble-be.classpath" | ||
| echo "[hubble] compiling BE with Java 11 + ${runner}" | ||
| ( | ||
| cd "${WORKTREE}/hugegraph-hubble" |
There was a problem hiding this comment.
hubble-be, while that module depends on the current hugegraph-loader/client artifacts and loader_home() later requires an ignored hugegraph-loader/apache-hugegraph-loader-* distribution. setup.sh only installs frontend dependencies, so a fresh clone has neither prerequisite (the existing Hubble CI explicitly installs client+loader first). Please build/install the required reactor modules and produce/select the current Loader distribution deterministically, or provision a declared artifact, and cover be-start from a clean checkout.
| echo "Use the 'Hubble BE Switch' action to replace it." >&2 | ||
| return 1 | ||
| fi | ||
| rm -f "${owner}" |
There was a problem hiding this comment.
kill_tree() only sends TERM; this loop waits for the wrapper shell PID, not the Java listener/descendants, so a slow graceful shutdown leaves an unowned listener and the subsequent port check rejects the switch. stop_current() has the same ownership-loss pattern without any wait. Please retain the owner until the captured process tree and port have exited (with a bounded timeout and non-zero failure), and add a delayed-TERM lifecycle test.
| compose_files low "${server_repo}" | ||
| echo "[infra] explicitly updating latest images" | ||
| TOOLCHAIN_CODEX_DIR="${CODEX_DIR}" HUGEGRAPH_VERSION=latest \ | ||
| docker compose "${COMPOSE_ARGS[@]}" pull |
There was a problem hiding this comment.
Infrastructure Pull currently skips every image because the low-memory overlay sets pull_policy: never. On this head, docker compose ... --dry-run pull reports Skipped for server/store/PD, while adding --policy always reports the expected pulls. Please explicitly override the service policy here (for example, pull --policy always) and test that this action really refreshes the images.
| - "--" | ||
| - /opt/toolchain-low-memory/patch-server-config.sh | ||
| environment: | ||
| PASSWORD: ${HUBBLE_SERVER_PASSWORD:-pa} |
There was a problem hiding this comment.
host_ip for 8080, 8520, or 8620, so the documented admin/pa Server and the Store/PD management endpoints are reachable from the developer's LAN. Please bind the necessary Server port to loopback, avoid publishing Store/PD to the host by default, and require or safely generate a local password.
| compose_files low "${server_repo}" | ||
| echo "[infra] stopping Compose services; named volumes are preserved" | ||
| TOOLCHAIN_CODEX_DIR="${CODEX_DIR}" HUGEGRAPH_VERSION=latest \ | ||
| docker compose "${COMPOSE_ARGS[@]}" down |
There was a problem hiding this comment.
down does not preserve the Server state as documented. hugegraph/server:latest declares VOLUME /hugegraph-server, but the merged Compose model does not attach a named volume there; removing the container leaves an anonymous volume that the next up will not reuse, including the initialization marker and local configuration. Please declare a stable named Server volume (and let reset remove it), or use docker compose stop for the preserving operation, with a stop/start persistence test.
| --add-exports=java.base/sun.nio.ch=ALL-UNNAMED | ||
| volumes: | ||
| - type: bind | ||
| source: ${TOOLCHAIN_CODEX_DIR}/infra/patch-server-config.sh |
There was a problem hiding this comment.
~/.codex/run/..., or package it in a stable image/volume, before starting the persistent service.
| grep -qx 'batch.max_write_threads=16' "${missing_yaml}/rest-server.properties" | ||
| } | ||
|
|
||
| test_environment_toml |
There was a problem hiding this comment.
environment_test.sh, and hubble-ci.yml does not watch .codex/**; the current green Hubble job ran because the README changed and never executed these tests. Please add a workflow/step triggered by .codex/** that runs this suite, shell/static checks, and both layered docker compose config validations, including the BE and infrastructure lifecycle cases above.
| python3 - "${CODEX_DIR}/environments/environment.toml" <<'PY' | ||
| import pathlib | ||
| import sys | ||
| import tomllib |
There was a problem hiding this comment.
bash .codex/tests/environment_test.sh command is not runnable with the target Mac's default Python: /usr/bin/python3 3.9.6 fails here with ModuleNotFoundError: No module named 'tomllib', while setup neither provisions nor checks Python 3.11+. Please use a compatible parser/fallback, or explicitly provision and validate Python 3.11+ in setup, documentation, and CI.
| owner_identity_matches "${file}" || return 1 | ||
| pid=$(read_field "${file}" pid) | ||
| port=$(read_field "${file}" port) | ||
| listener=$(lsof -tiTCP:"${port}" -sTCP:LISTEN 2>/dev/null | head -1 || true) |
There was a problem hiding this comment.
lsof is silently treated as 'no listener'; similarly, kill_tree() swallows a missing pgrep. On a host without these undeclared tools, the script can claim an occupied port or stop only the wrapper and orphan Java/Node children. Please validate these required commands during setup/start with an actionable error, or provide reliable portable fallbacks instead of converting command-not-found into an empty result.
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Adds a repository-managed Codex environment to make local Hubble development deterministic across worktrees, including optional HStore Compose profiles and verification actions.
Changes:
- Document optional Codex worktree workflow in the Hubble README.
- Add Codex environment definition plus setup/verify/dev-service management scripts.
- Add Compose overlays + server config patching script, with a shell test for key environment behaviors.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| hugegraph-hubble/README.md | Adds documentation entry point to the Codex environment guide. |
| .codex/README.md | Documents runtime model, ownership, HStore profiles, and safety notes. |
| .codex/environments/environment.toml | Defines Codex setup/cleanup plus FE/BE/infra/verify actions. |
| .codex/scripts/setup.sh | Installs FE deps and validates local toolchain (Node/Java/mvnd). |
| .codex/scripts/verify.sh | Adds fast/full verification entrypoints for FE + BE. |
| .codex/scripts/hubble.sh | Implements FE stable ports, BE ownership switching, and status/stop actions. |
| .codex/scripts/infra.sh | Adds HStore start/status/stop/reset/pull helpers and compose layering. |
| .codex/infra/compose.low-memory.yml | Provides low-memory Compose overrides and server entrypoint patching. |
| .codex/infra/compose.balanced.yml | Provides a “balanced” (less constrained) Compose override profile. |
| .codex/infra/patch-server-config.sh | Patches server configs for low-memory defaults before container entrypoint. |
| .codex/tests/environment_test.sh | Adds a bash test validating env TOML, port stability, compose args, and patching. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if [[ -n "${JAVA11_HOME:-}" ]] && [[ -x "${JAVA11_HOME}/bin/java" ]]; then | ||
| JAVA_HOME=${JAVA11_HOME} | ||
| elif [[ -x /usr/libexec/java_home ]]; then | ||
| JAVA_HOME=$(/usr/libexec/java_home -v 11) | ||
| elif command -v java >/dev/null 2>&1 && \ | ||
| java -version 2>&1 | head -1 | grep -Eq 'version "11\.'; then | ||
| JAVA_HOME=$(cd "$(dirname "$(command -v java)")/.." && pwd) | ||
| else | ||
| echo "ERROR: Java 11 is required; set JAVA11_HOME" >&2 | ||
| exit 1 | ||
| fi |
| python3 - "${CODEX_DIR}/environments/environment.toml" <<'PY' | ||
| import pathlib | ||
| import sys | ||
| import tomllib |
| hash_text() { | ||
| printf '%s' "$1" | LC_ALL=C shasum -a 256 | awk '{print substr($1, 1, 12)}' | ||
| } |
| command -v yarn >/dev/null || { | ||
| echo "ERROR: yarn is not available with Node 18.20.8" >&2 | ||
| exit 1 | ||
| } |
| if [[ -z "${container}" || "${project}" != hugegraph-single || \ | ||
| "${service}" != "${expected_container#hg-}" || \ | ||
| "${mapping}" != *":${port}"* ]]; then | ||
| echo "ERROR: port ${port} is owned by non-Compose pid ${listener}" >&2 | ||
| exit 1 | ||
| fi |
Summary
Add a repository-managed Codex environment for local Hubble development.
Scope
~/.codex/run/; no secrets or runtime artifacts are tracked.latestimages and does not pull automatically.Verification
bash .codex/tests/environment_test.shdocker compose ... config --quietfor low-memory and balanced profilesgit diff --check