test(install): add Docker lifecycle sandbox

[alacasse]

Hi @safishamsi,

Apologies for the delay, as per this conversation I created this PR for the docker sandbox validation for graphify installer. Feel free to ask any questions. I hope this could help the product.

Summary

This PR adds a Tier 1 Docker sandbox for Graphify’s installer. It exercises the real public CLI in isolated user and
project roots and verifies the Graphify-owned filesystem effects of installation, reinstallation, repair,
uninstallation, universal uninstallation, and purge.

This is intentionally a diagnostic PR: it does not modify graphify/install.py or include installer fixes. Its purpose
is to establish an independent, reproducible behavioral baseline before refactoring the installer.

What the sandbox adds

The sandbox currently covers 24 install targets and 50 target/scope scenarios. It verifies:

• exact installed payloads and version stamps;
• progressive reference bundles and repair of stale sidecars;
• owned Markdown sections and their complete bodies;
• owned JSON entries and hook payloads;
• reinstall idempotence;
• preservation of unrelated user files, Markdown, and JSON;
• removal of Graphify-owned state without broader directory deletion;
• absence of undeclared filesystem changes;
• universal user and project uninstall behavior;
• preservation of graphify-out/ without --purge;
• removal of graphify-out/ with --purge.

The source checkout is mounted read-only and copied to a separate installation root. HOME, XDG_CONFIG_HOME, the
project directory, user working directory, installed source, and diagnostic output are isolated from one another. The
real user home is never mounted.

Runs produce a readable report.md, machine-readable manifests and per-scenario results, command logs, filesystem
snapshots, and host-owned run.json/runner.log lifecycle diagnostics. Local managed runs use fresh directories and
safe keep-five retention.

The full Docker workflow is advisory while the unchanged installer has known behavioral failures: completed findings
are reported with artifacts and warnings, while incomplete, interrupted, malformed, or infrastructure-failed runs
fail the workflow.

The specs are central

Each YAML filename defines an install-target identity. The YAML owns target-specific facts such as supported scopes,
destinations, packaged payloads, owned markers, expected install surfaces, runtime limitations, and the current
universal-uninstall eligibility.

The strict loader owns schema validation, defaults, and deterministic derivation. The lifecycle code operates
generically over the loaded catalog; it does not maintain a second Python list or grouping of real target names.

The specs are also owned by the Docker image rather than loaded from the repository under test. This prevents a
product change from redefining its own expected result and making a defect pass its test.

The current YAML shape is not intended to be permanent. I would continue improving it by:

• separating destination facts, executable recipe facts, ownership/cleanup identities, and independent assertions;

• deriving values from filenames, scope, existing facts, or target-independent conventions whenever that is safe;

• keeping validation order, reporting, purge behavior, and other cross-target harness policy in generic Python;

• replacing transitional command-mode and universal-uninstall declarations when those decisions can be derived
  safely;

• describing complete generated Markdown and JSON payloads rather than relying only on required fragments;

• adding precise current and legacy ownership identities and cleanup-only destinations.

The architectural goal is that adding a supported tool becomes a spec-file change, plus any payload referenced by
that spec—not another collection of target tables, dispatch branches, and cleanup lists scattered through Python.

In this first PR, the specs drive the sandbox and are checked against the existing public target list. Parts 2 and 3
below would make them the production installer’s source of truth as well.

Installer findings

Running the harness against unchanged v8 exposed six product-installer defects. The twelve failing scenario rows in
the clean full-catalog run map to these six root causes; they are not twelve different bugs.

Finding	Observable behavior
CodeBuddy scope inconsistency	CodeBuddy artifacts do not consistently remain within the selected user or project scope.
Progressive sidecars survive uninstall	SKILL.md can be removed while Graphify-owned references/, references.tmp, or .graphify_version artifacts remain.
Claude user uninstall leaves its registration	The user skill can be removed while the owned registration remains in ~/.claude/CLAUDE.md.
Antigravity discovery metadata is omitted	The installed skill can retain generic frontmatter instead of Antigravity’s native discovery name and description.
Universal user uninstall is incomplete	graphify uninstall leaves Graphify-owned user integrations at multiple destinations.
Stale Claude and Windows registrations are not upgraded	Any existing registration containing the word graphify can prevent replacement with the current scope-correct registration.

None of these fixes is included here, and the oracle has not been weakened to accept them.

If you agree with the classifications, I can open separate GitHub issues containing compact reproductions and
acceptance criteria, and provide independently reviewable fixes. I already have isolated fixes prepared for the first
four findings. I would address stale registration replacement before universal uninstall because it otherwise
interferes with clean aggregate setup and repair observations.

Validation

The harness-only branch is one commit above the verified v8 base and changes only the sandbox tooling, sandbox tests,
target specs, and scoped CI workflows.

Validation completed on the clean branch:

• focused sandbox suite: 104 passed;
• Ruff passed;
• targeted Pyright: 0 errors, 0 warnings;
• workflow YAML parsing and git diff --check passed;
• complete frozen suite on Python 3.10 and 3.12: 3,899 passed, 3 skipped on each;
• full catalog: PASS=37, FAIL=12, UNSUPPORTED=1; purge passed.

Reproduction:

uv run --frozen python tools/install_sandbox/run.py
--repo .
--all
--scope both

The sandbox proves Graphify-owned filesystem effects. It does not prove that a target application is installed,
authenticates, discovers the integration, loads it, or executes it. Windows scenarios in the Linux container verify
packaged payload consistency rather than native Windows lifecycle behavior.

CI/CD integration and developer impact

The sandbox is integrated at two complementary CI levels.

First, the existing test workflow keeps the fast checks blocking:

• normal pytest loads the real YAML catalog and verifies that its filename-derived targets match the public graphify
  install --help target list;

• tools/install_sandbox is type-checked with Pyright;

• these checks run with the existing CI suite, so missing specs, stale specs, invalid schema, and sandbox typing
  regressions are caught quickly.

Second, a dedicated Install sandbox workflow runs the full Docker filesystem-effect validation as one Ubuntu/Python
3.12 job:

uv run --frozen python tools/install_sandbox/run.py
--repo .
--all
--scope both
--output "$RUNNER_TEMP/graphify-install-sandbox"

It runs:

• on pull requests targeting v8 or main when installer code, packaged skills or commands, sandbox code or tests,
  packaging metadata, uv.lock, or the workflow itself changes;

• after pushes to v8 or main affecting those same paths;

• nightly at 05:27 UTC;

• manually through workflow_dispatch.

Unrelated changes do not start the Docker workflow. It is also kept outside the Python-version test matrix, so a
relevant change adds one full-catalog Docker job rather than repeating the matrix for every Python version. When a
pull request is updated, an older in-progress sandbox run for that PR is cancelled.

The workflow always preserves its diagnostic output, including when the sandbox reports failures. It uploads the
complete run directory under a unique run/attempt name, retains it for 14 days, and uses maximum compression. The
GitHub Actions job summary contains:

• the original sandbox exit code;
• the generated report.md, when available;
• a fallback to run.json if behavioral execution did not complete;
• a direct link to the complete diagnostic bundle.

The result is deliberately classified rather than treating every nonzero sandbox exit as the same kind of failure:

• a completed passed run succeeds;
• a completed run containing product findings emits a warning and also succeeds;
• an incomplete or interrupted run fails;
• missing or malformed lifecycle metadata fails;
• disagreement between the recorded runner exit code and run.json fails.

This distinction matters while the unchanged installer still has known behavioral defects. Developers can see and
download the exact failed scenarios without those already-classified product findings blocking unrelated work.
However, failures in the sandbox itself—Docker setup, catalog loading, interrupted execution, missing evidence, or
inconsistent result metadata—remain visible CI failures.

For developers changing the installer, this adds an automatic cross-target regression report with no special local
setup required beyond the existing PR workflow. For developers adding a target, the fast catalog check requires the
public installer target and its spec to remain synchronized. The nightly run then provides continuing full-catalog
coverage even when no qualifying pull request or push has run recently.

Proposed follow-up

Part 2: production-oriented specs and a write-free plan

The next part would complete the YAML contract without changing the filesystem. The loader would normalize a target,
scope, package version, and installer options into a deterministic InstallPlan.

That plan would include complete payloads, destinations, ownership and removal identities, legacy replacements,
cleanup-only locations, runtime placeholders, path-resolution inputs, option variants such as strict mode, and
progressive-reference manifests.

The legacy CLI would remain the system being exercised, while the independent sandbox would compare its observed
effects with the normalized plan. This keeps schema review separate from mutation logic and proves that no
undisclosed target-specific behavior remains hidden in the legacy installer.

Part 3: Installer Core and CLI cutover

Once the plans are complete and independently validated, the final part would add the generic Installer Core that
applies them.

This part would own filesystem and structured-file mutation, transaction and rollback boundaries, symlink/permission/
encoding/invalid-input policy, error reporting, public CLI integration, and removal of superseded target-specific
installer paths.

The existing sandbox would remain independent and become the cutover acceptance contract: replacing the legacy
implementation with the new executor should preserve intended behavior, while deliberate behavior changes remain
explicit.

The result would be one target catalog feeding a generic loader, planner, installer, uninstaller, and behavioral
sandbox—so adding a new tool is primarily a matter of adding its spec.