From cf92800522a3c6959c20412ffd4329a8b4068d3f Mon Sep 17 00:00:00 2001 From: Matei Date: Fri, 12 Jun 2026 02:57:50 -0400 Subject: [PATCH] =?UTF-8?q?docs(mapgen-studio):=20effect-native=20substrat?= =?UTF-8?q?e=20spike=20=E2=80=94=20verdict:=20keep=20the=20hand-rolled=20s?= =?UTF-8?q?hell,=20revisit=20at=20Effect=20v4=20stable?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three-investigator research wave (platform-bun HTTP/runMain; Socket/ Command/Stream; repo deletion ledger + effect-orpc interplay), verified against published tarball source. All three rewrite candidates fail the leverage falsifier (not simpler than what they delete): daemon shell REJECT (adapter ≈ deleted code, HEAD/stop(true) drift traps, 5+ dep surface, double runtime survives), tuner client rewrite REJECT (relocation not deletion; platform Socket mismatches the soak-verified FIN/reconnect invariants), devLive Command PARTIAL-deferred. Revisit triggers recorded: Effect v4 latest dist-tag (platform folds into core), WebSockets/caching needs, frame-path pressure (Mailbox/Deferred inside the class boundary). Co-Authored-By: Claude Fable 5 --- .../mapgen-studio-redesign/00-GOAL.md | 12 ++ .../04-effect-native-substrate-spike.md | 137 ++++++++++++++++++ 2 files changed, 149 insertions(+) create mode 100644 docs/projects/mapgen-studio-redesign/research/04-effect-native-substrate-spike.md diff --git a/docs/projects/mapgen-studio-redesign/00-GOAL.md b/docs/projects/mapgen-studio-redesign/00-GOAL.md index 2bcf8680c6..ecaefcf0d2 100644 --- a/docs/projects/mapgen-studio-redesign/00-GOAL.md +++ b/docs/projects/mapgen-studio-redesign/00-GOAL.md @@ -450,3 +450,15 @@ addressed to this lane sustained polling, zero leaked fds, game releases its descriptor on daemon stop. Run-in-game flows untouched (parity). Deferred: run-flow convergence; "restart Civ7" UI affordance. +- [x] **Effect-native substrate spike** — user-directed leverage question: should + the daemon shell, devLive runner, and tuner protocol client be rebuilt on + `@effect/platform(-bun)` primitives (HttpServer/runMain, Command, + Socket/Stream/Deferred)? Three-investigator research wave, verified against + published tarball source. **Verdict: REJECT all three; revisit at Effect v4 + stable** — adapters ≈ deleted code, platform-bun drags non-optional + rpc/sql/cluster peers, platform Socket semantics mismatch the soak-verified + FIN/reconnect invariants, effect-orpc requires its own ManagedRuntime + regardless (no "one graph" payoff), and v4 (in beta) folds platform into + effect core — adopting v3 platform now buys a guaranteed second migration. + Full evidence + revisit triggers: + `research/04-effect-native-substrate-spike.md`. diff --git a/docs/projects/mapgen-studio-redesign/research/04-effect-native-substrate-spike.md b/docs/projects/mapgen-studio-redesign/research/04-effect-native-substrate-spike.md new file mode 100644 index 0000000000..b34b2b6403 --- /dev/null +++ b/docs/projects/mapgen-studio-redesign/research/04-effect-native-substrate-spike.md @@ -0,0 +1,137 @@ +# Effect-native substrate spike — VERDICT: keep the hand-rolled shell (revisit at Effect v4 stable) + +**Date:** 2026-06-12 (post tuner-session workstream, stack tip +`design/tuner-daemon-wiring`). + +**Question (user-directed):** the daemon (`daemon.ts`), dev runner +(`devLive.ts`), and tuner protocol client +(`Civ7DirectControlSession`) hand-roll machinery Effect provides +natively — buffering (`Stream`/`Mailbox`), scope/release, HTTP serving +(`@effect/platform-bun`), child processes (`Command`), request +correlation (`Deferred`), timeouts, connect dedup. Would replacing them +with Effect-native primitives be simpler AND more robust — the leverage +principle, "a solution to a problem that goes away when you stop +creating that problem"? Standing falsifier: **if a native rewrite is +not simpler than what it deletes, that's a verdict, not a failure.** + +**Method:** three parallel investigators, all verifying against +published tarball source (not memory): (A) `@effect/platform-bun` HTTP +serving + `runMain`; (B) platform `Socket`/`Command`/Stream framing for +the tuner client and devLive; (C) repo deletion ledger + +stable-surface constraints + effect-orpc interplay. + +## The falsifiers fired. All three candidates rejected or deferred. + +### Ecosystem facts (verified 2026-06-12, npm registry + tarballs) + +- `effect@3.21.3` (our workspace pin) **is the current stable** — no + bump needed for anything. +- `@effect/platform@0.96.1` / `@effect/platform-bun@0.90.0` are + peer-compatible (`effect ^3.21.2`), BUT platform-bun declares + **non-optional peers** `@effect/rpc`, `@effect/sql`, + `@effect/cluster` (no `peerDependenciesMeta`) — bun installs all of + them. 5+ new packages for one HTTP shell. +- **Effect v4 is in beta** (`4.0.0-beta.80`, published 2026-06-11; + official guidance: "v3 remains our recommended choice for + production"). In v4, `@effect/platform`/`rpc`/`cluster` are **folded + into effect core** (`effect/unstable/*` paths; verified + `@effect/platform-bun@4.0.0-beta.80` has a single `effect` peer). + Anything written against `@effect/platform@0.96.x` today is a + guaranteed second migration. + +### 1. Daemon HTTP shell → `BunHttpServer` + `runMain`: **REJECT** + +- LOC accounting is a wash: ~150 deleted (router, signals, Bun.serve + wiring, mime/static) vs ~100–120 added (HttpRouter graph, a + `fromOrpc` passthrough helper, Layer/scope plumbing) — plus the 5+ + dep surface. +- The load-bearing invariant (byte-identical oRPC mount behavior) is + exactly where the drift risk lands: `HttpApp.fromWebHandler` + decomposes/rewraps the Response (framing can drift); the correct + idiom is `HttpServerRequest.toWeb` + `HttpServerResponse.raw` + (zero-copy on Bun, verified), but Bun's adapter **short-circuits HEAD + requests** before the raw branch, and the server-layer finalizer is + graceful `server.stop()` (a wedged in-flight request blocks shutdown) + vs today's deliberate `stop(true)`. +- Shutdown ordering inverts: Layer finalizers run server-stop **then** + runtime dispose (tuner FIN); today we FIN first, then `stop(true)`. + Changeable, but it's a behavior change on the fd-leak-mitigation + path, not a free win. +- **The "one Effect graph" payoff is unattainable:** effect-orpc 0.2.2 + takes a `ManagedRuntime` positionally (`implementEffect(contract, + runtime)`) and executes every procedure on it via + `runtime.runPromiseExit`. A `runMain`-launched Layer graph would be a + *second* runtime; the double-runtime topology survives any rewrite. + (`ManagedRuntime.make(layer, memoMap)` exists for cross-runtime layer + sharing if ever needed.) +- v3 platform static serving gives `HttpServerResponse.file` (etag + emitted, Bun.file streaming) but **no 304/if-none-match handling, no + path jail, no SPA fallback** — our ~25-line jail/fallback survives + regardless. +- `@effect/cli` for three flags: overkill (drags printer peers). + +### 2. Tuner protocol client rewrite (platform Socket + Stream + Deferred): **REJECT** + +- "Relocation, not deletion": deletes ~115 lines of tested mechanism in + `session.ts`, recreates **≥130 lines** of Effect mechanism (Deferred + map, drain fiber, mapAccum framer, host fallback, lazy + reconnect-on-next-request supervision, error-code mapping, custom + FIN finalizer) **plus a promise facade** (`ManagedRuntime` + + `runPromise` per method), because the byte-stable promise API (~60 + procedures, CLI + control-orpc + engines + proof subpath consumers, + CJS build) and the zero-effect-dep constraint force an adapter no + matter where the Effect core lives. +- Platform Socket semantics **mismatch our soak-verified invariants**: + `makeNet` dials lazily inside each `run` (reconnect idiom = re-run + `run`, not reconnect-on-next-request); `writer` is latched on an + active reader fiber (write-without-reader suspends forever); the + scope finalizer is `destroySoon()`, **not** our bounded FIN→await + close→destroy handshake — which is the suspected fd-leak mitigation, + pinned by tests. +- No built-in length-prefix framing exists (Ndjson/MsgPack are the + precedents; `Stream.mapAccum` rebuilds our loop in the same line + count — the pure byte arithmetic in `framing.ts` is irreducible and + already factored out). +- For request/response correlation the right primitive is `Deferred` + per listenerId (not Queue/Mailbox — those are ordered pipes; + `Mailbox` would only fit the single raw-bytes hop). +- **The architecture already has Effect at the right altitude:** + `Civ7TunerSession` (Layer.scoped + acquireRelease + backoff gate + + health) owns lifecycle/policy; the class owns bytes. The gate's + out-of-band `session.stats` read is deliberate — it is the one + vantage that sees ALL shared-socket traffic including the + control-oRPC mount, which bypasses `use()`. Moving the counter into a + typed error channel inside the gate would *lose* that visibility. +- If frame-path pressure ever returns, the surgical option is + `Mailbox` + `Deferred` *inside* the existing class boundary — not a + transport rewrite. + +### 3. devLive → platform `Command`: **PARTIAL, deferred** + +The honest inverse case: `Command.start` is a verified +`acquireRelease` that **SIGTERMs the process group** on scope close +(catches vite's esbuild grandchildren — strictly stronger than our +`child.kill()`), and `Effect.raceFirst` + `Schedule.spaced` genuinely +delete ~60–80 lines of supervision (signal handlers, stop-all flag, +first-exit promise, readiness poll). But it's a dev-only script that +already works, and the price is the same `@effect/platform*` dep +surface rejected above. Verdict: adopt **only** when the platform +packages enter the workspace for other reasons. + +## Revisit triggers (recorded, not scheduled) + +1. **Effect v4 hits the `latest` dist-tag** — platform moves into + `effect` core (which we already depend on), the peer-weight + objection dissolves, and the `runMain` + scoped-finalizer shutdown + shape becomes worth taking. Re-run this spike then. +2. Wanting WebSockets on the daemon (platform-bun's upgrade support is + genuinely good) or real HTTP caching/range asset serving. +3. Frame-path pressure in the tuner client → `Mailbox`/`Deferred` + inside the class boundary, never a transport rewrite. + +## Standing regression gate (unchanged) + +Live soak on `:4318`: exactly 1 ESTABLISHED connection across +sustained polling, zero CLOSED fds, descriptor released on daemon +stop; healthz tuner block clean. Test pins: direct-control 390, studio +204, mod 471.