Add engine-only launch smoke test on macos-15 CI (needs no retail data)

[bobtista]

Problem

There is no CI coverage of the native launch path. Launch regressions surface only manually, on a developer machine, after the fact.

Corrected premise — read before implementing

This issue originally said the test "needs no retail data (engine reaches main loop with no content)". The engine cannot reach its main loop with no content. Verified in INI.cpp:222:

// Expect to open and load at least one file.
if (filesRead == 0)
{
    throw INI_CANT_OPEN_FILE;
}

INI_CANT_OPEN_FILE is in an unnamed enum, so it throws a plain int, does not match catch (ErrorCode ec), and falls to catch (...) → RELEASE_CRASH → _exit(1). All of that is in upstream TheSuperHackers/GeneralsGameCode@main, not fork-local — observed behaviour, though not design intent: the routing is accidental (catch (ErrorCode ec) acts only on ERROR_INVALID_D3D and silently swallows anything else, so the plain-int route to catch (...) is what actually produces the crash). Assert what is observed, not that it was intended.

So the test as originally specified cannot be built. It is still worth building, asserting the opposite outcome.

What to assert instead

Launch the engine with no retail data and assert it exits 1 with the initialisation-failure diagnostic. That still validates a great deal, and needs no licensed content:

• the binary loads at all
• dynamic libraries resolve via @executable_path / RPATH — the failure mode Remove DYLD_LIBRARY_PATH and LD_LIBRARY_PATH from native launch #317 was about
• initialisation runs far enough to attempt INI loading
• the documented failure path is reachable rather than hanging

A test that reaches the main loop needs content and therefore cannot run in CI. That is a separate, local concern.

Flags: check fork status before relying on one

Flag	Upstream main	bgfx trunk
-headless	yes	yes
-noDraw, -noaudio, -quickstart, -win	yes	yes
-perfAutoExitSeconds	no	yes

Use -headless — it exists in both. Do not build this test on -perfAutoExitSeconds: it is fork-only, so a test depending on it validates our client and breaks against upstream.

-perfAutoExitSeconds is still useful locally: the three real-client tests currently launch the game and hold it for 14 seconds, which is why they fail whenever a developer closes the window. Self-termination would fix that, and is fine there because those tests only ever run against our client.

Acceptance criteria

• A macos-15 CI job launches the engine with no retail data and asserts exit 1 plus the expected diagnostic.
• No licensed game content in CI.
• Runtime is bounded, and the job fails rather than hangs if the engine neither exits nor produces the diagnostic.
• No dependency on fork-only flags.

Related

• Remove DYLD_LIBRARY_PATH and LD_LIBRARY_PATH from native launch #317 — loader search path; this test is what would catch a regression in it
• Zero Hour can still launch with no base Generals content when no Generals root is declared #333 — the archive failure the engine may not report (silent skip verified in source; end-to-end behaviour not yet reproduced)
• Detect engine archive failures after spawn: 500 ms window and unused [ggc] stderr sentinel #334 — GenHub-side detection of the failures it does report

[coderabbitai]

Coding Plan

Summary

• Add a dedicated, self-contained bash smoke script (scripts/ci/) that downloads the published macOS engine build (latest-bgfx → GeneralsZH-macos-arm64.zip), launches it headless with no content, and asserts a bounded exit code 1 with a best-effort init-failure diagnostic check, failing on hang or unexpected exit.
• Introduce a macos-15 job into .github/workflows/ci.yml following the existing detect-changes-gated / summary-aggregated conventions, invoking the script and adding a job-level timeout backstop.
• Depend only on the cross-fork -headless flag and avoid all licensed content, satisfying every acceptance criterion; explicitly leave the local -perfAutoExitSeconds real-client-test change out of scope.
• Add a short doc note formalizing what the smoke test validates (including the Remove DYLD_LIBRARY_PATH and LD_LIBRARY_PATH from native launch #317 loader-path coverage).

Design Choices

Design Choice 1: How strictly should the "expected diagnostic" be asserted?

Options Considered:

1. Make a specific stderr/console string match the hard pass/fail gate.
2. Make exit code 1 within a bounded time the hard gate, plus a best-effort search of captured output and crash-dump locations for the diagnostic (logged, and used to strengthen the pass), failing the job on hang or on a non-1 exit.

Chosen Option: 2

Rationale: Option 2 satisfies all acceptance criteria robustly (exit 1, bounded, fail-not-hang) without depending on an unverified log string, while still checking for the diagnostic when available. Web research shows the engine's early-init logging may not reach stdout/stderr reliably, and crash dumps write to a Documents path; the exact console string for the INI_CANT_OPEN_FILE → RELEASE_CRASH path is unverified.

Design Choice 2: Should the smoke logic be inline in `ci.yml` or a dedicated script?

Options Considered:

1. Inline bash inside the job's run: block, matching current convention.
2. A dedicated versioned bash script invoked by the job.

Chosen Option: 2

Rationale: A dedicated script is far more readable/maintainable/testable for this multi-step routine; it establishes a small, clearly-scoped CI-scripts convention. Existing jobs embed logic inline, but this routine (download, verify, bounded launch, exit capture, grep) is non-trivial.

Design Choice 3: Which engine artifact/release to fetch?

Options Considered:

1. Latest tagged main release assets.
2. The latest-bgfx release asset GeneralsZH-macos-arm64.zip.

Chosen Option: 2

Rationale: It is the published macOS (Apple Silicon) build with bundled dylibs, matching macos-15's arm64 runner and directly exercising the @executable_path/RPATH resolution the test targets.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Engine launch smoke-test script

Create a self-contained, bounded smoke routine that fetches the macOS engine build, launches it headless with no content, and asserts the intended initialization-failure outcome. This phase encapsulates all the tricky logic (download, RPATH sanity, bounded wait, exit-code capture, diagnostic search) in one reviewable unit, following the bash idioms already used in the repo's update script.

Add the macOS engine smoke-test script

Create the script that performs download → launch → assertion, independent of any GenHub .NET code.

• Create a new bash script in a new scripts/ci/ directory (e.g., scripts/ci/macos-engine-smoke-test.sh); follow the repo's bash conventions (#!/bin/bash, set -euo pipefail, a timestamped log function, explicit if ! <cmd>; then ... exit 1; fi checks, defensive cleanup of temp dirs).
• Resolve and download the macOS engine artifact from the thesuperhackers/GeneralsGameCode latest-bgfx release (asset GeneralsZH-macos-arm64.zip) via curl against the GitHub Releases API; treat the owner/repo values as mirroring GenHub.Core/Constants/SuperHackersConstants.cs. Fail clearly if the asset cannot be located or downloaded.
• Extract into a fresh temporary working directory that contains no retail INI/content files, ensuring the engine hits the empty-content path.
• Before launch, run otool -L/otool -l on the engine binary and log the LC_RPATH / @executable_path entries — this documents and lightly guards the Remove DYLD_LIBRARY_PATH and LD_LIBRARY_PATH from native launch #317 loader-search-path concern.
• Launch the binary with only the -headless flag (never -perfAutoExitSeconds or other fork-only flags), from the empty working directory, redirecting stdout+stderr to a captured log.
• Implement a self-contained bounded wait (poll the PID in a loop with a fixed max duration; do not rely on timeout/gtimeout, which are absent on macOS runners). If the process must be force-killed, treat it as a hang and fail with a clear message.
• Assert the captured exit code equals 1; fail on any other exit code. As a secondary check, grep the captured log and known crash-dump locations for the initialization-failure diagnostic (e.g., INI_CANT_OPEN_FILE / RELEASE_CRASH / crash-dump presence), logging the result per Assumption 1.
• Emit a concise pass/fail summary and exit non-zero on any failure so the CI step reflects the outcome.

🤖 Prompt for AI agents

Implement Phase 1: create a self-contained bash smoke-test script that downloads
the macOS engine build, launches it headless with no retail content, and asserts
the initialization-failure outcome (exit code 1, bounded, fail-not-hang).

- Create a new `scripts/ci/` directory and add
`scripts/ci/macos-engine-smoke-test.sh`. Follow repo bash conventions:
`#!/bin/bash`, `set -euo pipefail`, a timestamped log function, explicit `if !
<cmd>; then ... exit 1; fi` checks, and defensive cleanup of temp directories.
- Resolve and download the macOS engine artifact from the
`thesuperhackers/GeneralsGameCode` `latest-bgfx` release, asset
`GeneralsZH-macos-arm64.zip`, using `curl` against the GitHub Releases API.
Mirror owner/repo values from `GenHub.Core/Constants/SuperHackersConstants.cs`.
Fail clearly with a message if the asset cannot be located or downloaded.
- Extract the artifact into a fresh temporary working directory containing no
retail INI/content files, so the engine hits the empty-content path.
- Before launching, run `otool -L` and `otool -l` on the engine binary and log
the `LC_RPATH` / `@executable_path` entries as a documentation/guard step for
the #317 loader-search-path concern.
- Launch the binary using only the `-headless` flag (do not use
`-perfAutoExitSeconds` or any other fork-only flag), running it from the empty
working directory, and redirect stdout+stderr into a captured log file.
- Implement a self-contained bounded wait by polling the process PID in a loop
with a fixed maximum duration; do not use `timeout`/`gtimeout` since these are
unavailable on macOS runners. If the process needs to be force-killed after the
bound, treat this as a hang and fail with a clear message.
- After the process exits (or is killed), assert the captured exit code equals
1; fail on any other exit code. As a secondary, best-effort check, grep the
captured log and known crash-dump locations for the initialization-failure
diagnostic (e.g., `INI_CANT_OPEN_FILE`, `RELEASE_CRASH`, or crash-dump file
presence) and log whether it was found, without making it the hard gate.
- Emit a concise pass/fail summary at the end of the script and exit non-zero on
any failure condition so the CI step reflects the true outcome.

Research

GenHub is a .NET/Avalonia launcher that orchestrates externally-installed retail games; it does not vendor, build, or test against the native engine, which lives in the separate thesuperhackers/GeneralsGameCode repo (referenced only by SuperHackersConstants.cs). CI is a single .github/workflows/ci.yml with a detect-changes paths-filter job feeding build-windows / build-linux, aggregated by a summary job; there is no macOS job, no build matrix, and no shared scripts/ directory (job logic is inline bash/pwsh). This work is net-new: it adds an engine launch smoke test that has no prior pattern in the repo.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Implement Phase 1: create a self-contained bash smoke-test script that downloads
the macOS engine build, launches it headless with no retail content, and asserts
the initialization-failure outcome (exit code 1, bounded, fail-not-hang).

- Create a new `scripts/ci/` directory and add
`scripts/ci/macos-engine-smoke-test.sh`. Follow repo bash conventions:
`#!/bin/bash`, `set -euo pipefail`, a timestamped log function, explicit `if !
<cmd>; then ... exit 1; fi` checks, and defensive cleanup of temp directories.
- Resolve and download the macOS engine artifact from the
`thesuperhackers/GeneralsGameCode` `latest-bgfx` release, asset
`GeneralsZH-macos-arm64.zip`, using `curl` against the GitHub Releases API.
Mirror owner/repo values from `GenHub.Core/Constants/SuperHackersConstants.cs`.
Fail clearly with a message if the asset cannot be located or downloaded.
- Extract the artifact into a fresh temporary working directory containing no
retail INI/content files, so the engine hits the empty-content path.
- Before launching, run `otool -L` and `otool -l` on the engine binary and log
the `LC_RPATH` / `@executable_path` entries as a documentation/guard step for
the #317 loader-search-path concern.
- Launch the binary using only the `-headless` flag (do not use
`-perfAutoExitSeconds` or any other fork-only flag), running it from the empty
working directory, and redirect stdout+stderr into a captured log file.
- Implement a self-contained bounded wait by polling the process PID in a loop
with a fixed maximum duration; do not use `timeout`/`gtimeout` since these are
unavailable on macOS runners. If the process needs to be force-killed after the
bound, treat this as a hang and fail with a clear message.
- After the process exits (or is killed), assert the captured exit code equals
1; fail on any other exit code. As a secondary, best-effort check, grep the
captured log and known crash-dump locations for the initialization-failure
diagnostic (e.g., `INI_CANT_OPEN_FILE`, `RELEASE_CRASH`, or crash-dump file
presence) and log whether it was found, without making it the hard gate.
- Emit a concise pass/fail summary at the end of the script and exit non-zero on
any failure condition so the CI step reflects the true outcome.

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- @coderabbitai You can skip phase 3. Add a simple unit test case for phase 2.
- @coderabbitai For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[coderabbitai]

I've updated the plan above based on your feedback. Ready to proceed when you are!

[coderabbitai]

I've updated the plan above based on your feedback. Ready to proceed when you are!

[coderabbitai]

I've updated the plan above based on your feedback. Ready to proceed when you are!