Skip to content

Commit 3f8dca1

Browse files
vanceingallsclaude
andcommitted
fix(cli,core): refresh telemetry posture at the render boundary
R6/R7 blockers. An already-open Studio kept emitting server-side render telemetry after another process disabled CLI telemetry. refreshTelemetryPosture() only ran while serving a fresh SPA document and on /api/telemetry-identity, which Studio has no consumer for, so the render POST and its async outcome used the posture cached when the preview server booted. It now refreshes at the render boundary and again immediately before the completion/error event, so an opt-out during a long render is honoured. The identity tests were passing vacuously: their mocks omitted readConfigFresh and resetTelemetryPostureCache, and the resulting missing-export error was swallowed by the refresh's own catch. Mocked properly, plus the enabled -> external disable -> next response transition and the suppression path at the layer that drops the event. A full reset also did not persist its new lineage in a long-lived process: syncInstallState returned early on a process-lifetime memo even after ~/.hyperframes was deleted, so install-state was never recreated and the next config-only re-mint rolled a third seed instead of inheriting the second. The memo is now revalidated against the file. Also drops a stale reference to assertNoOverdueCanaries and stops the workflow and docs claiming the sunset job routes anything to the owner — it names them in the run log and notifies nobody. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent cad6b39 commit 3f8dca1

8 files changed

Lines changed: 174 additions & 6 deletions

File tree

‎.github/workflows/canary-sunset.yml‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,12 @@
22
#
33
# Deliberately NOT a PR gate. The check reads the current date, so as a PR gate
44
# it would fail builds for authors who touched nothing related, on a calendar
5-
# date, with no fix available to them. On a schedule the failure lands on the
6-
# rollout's owner instead, which is who can actually ramp it to 100 and delete
7-
# the guard.
5+
# date, with no fix available to them. On a schedule the failure stands on its
6+
# own instead of blocking an unrelated author.
7+
#
8+
# The job names the overdue canary and its owner in the run log; it does not
9+
# notify anyone. Routing that to the owner automatically (an issue, a ping)
10+
# is worth doing and is not done here.
811
name: Canary sunset
912

1013
permissions:

‎docs/contributing/canary-rollouts.mdx‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,8 @@ call site.
5656
**4. Delete it** once it is at 100 and holding — both the registry entry and
5757
the branch it guarded. `sunsetAfter` exists to force this: the scheduled
5858
**Canary sunset** workflow runs weekly and fails once the date passes, naming
59-
the rollout and its owner.
59+
the overdue rollout and its owner in the run log. It does not notify anyone —
60+
watch the workflow if you own a canary.
6061

6162
It is a scheduled job rather than a PR check on purpose. A current-date
6263
assertion in the unit suite would redden builds for authors who changed

‎packages/cli/src/server/studioServer.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -387,6 +387,10 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
387387
rendersDir: () => join(projectDir, "renders"),
388388

389389
startRender(opts): RenderJobState {
390+
// The render POST is a request boundary like any other. Without this an
391+
// already-open Studio tab keeps rendering under the posture cached when
392+
// the server booted.
393+
refreshTelemetryPosture();
390394
const abortController = new AbortController();
391395
const state: RenderJobState = {
392396
id: opts.jobId,
@@ -466,6 +470,12 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
466470
metaPath,
467471
JSON.stringify({ status: "complete", durationMs: Date.now() - startTime }),
468472
);
473+
// Refreshed HERE, not just at render start: a render can run for
474+
// minutes, and `hyperframes telemetry disable` during one must be
475+
// honoured by the event that reports it. Studio never polls
476+
// /api/telemetry-identity, so this process would otherwise keep its
477+
// startup-cached posture for the life of the preview server.
478+
refreshTelemetryPosture();
469479
emitStudioRenderComplete(opts, Date.now() - startTime, job.perfSummary);
470480
} catch (err) {
471481
if (abortController.signal.aborted) {
@@ -475,6 +485,7 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
475485
state.status = "failed";
476486
state.error = err instanceof Error ? err.message : String(err);
477487
// fallow-ignore-next-line code-duplication
488+
refreshTelemetryPosture();
478489
emitStudioRenderError(opts, Date.now() - startTime, state.stage, err, renderJob);
479490
try {
480491
const metaPath = opts.outputPath.replace(/\.(mp4|webm|mov)$/, ".meta.json");

‎packages/cli/src/server/telemetryIdentity.test.ts‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,20 @@ const readConfig = vi.fn();
1111
// don't move every time a canary is added, ramped, or retired.
1212
const canaryDecisions = vi.fn<() => Record<string, { enabled: boolean; forced: boolean }>>();
1313

14+
// Every export the module under test imports must be mocked. Omitting
15+
// `resetTelemetryPostureCache` / `readConfigFresh` made `refreshTelemetryPosture`
16+
// throw a missing-export error that its own catch swallowed, so every
17+
// assertion below ran against a refresh that silently did nothing.
18+
const resetPostureCache = vi.fn();
19+
const readConfigFresh = vi.fn();
20+
1421
vi.mock("../telemetry/client.js", () => ({
1522
shouldTrack: (...args: unknown[]) => shouldTrack(...args),
23+
resetTelemetryPostureCache: () => resetPostureCache(),
1624
}));
1725
vi.mock("../telemetry/config.js", () => ({
1826
readConfig: (...args: unknown[]) => readConfig(...args),
27+
readConfigFresh: () => readConfigFresh(),
1928
}));
2029
vi.mock("../telemetry/canary.js", () => ({
2130
canaryDecisionsForStudio: () => canaryDecisions(),
@@ -27,6 +36,7 @@ const {
2736
buildStudioHeadScripts,
2837
isLoopbackHost,
2938
buildStudioHeadScriptsForHost,
39+
refreshTelemetryPosture,
3040
identityAllowed,
3141
} = await import("./telemetryIdentity.js");
3242

@@ -344,3 +354,42 @@ describe("identityAllowed — loopback-bound vs explicitly LAN-bound", () => {
344354
});
345355
});
346356
});
357+
358+
// A long-lived preview server: the posture it cached at boot must not outlive
359+
// an opt-out run in another terminal. Studio has no poller for
360+
// /api/telemetry-identity, so the refresh has to happen on the paths that
361+
// actually run — the SPA document and the render boundary.
362+
describe("cross-process opt-out refresh", () => {
363+
beforeEach(() => {
364+
resetPostureCache.mockClear();
365+
readConfigFresh.mockClear();
366+
});
367+
368+
it("actually invalidates both caches — the mocks used to swallow this", () => {
369+
refreshTelemetryPosture();
370+
expect(readConfigFresh).toHaveBeenCalledTimes(1);
371+
expect(resetPostureCache).toHaveBeenCalledTimes(1);
372+
});
373+
374+
it("refreshes before building a head script", () => {
375+
shouldTrack.mockReturnValue(true);
376+
readConfig.mockReturnValue({ anonymousId: "id-1", bucketSeed: "seed-1" });
377+
canaryDecisions.mockReturnValue({});
378+
buildStudioHeadScriptsForHost("", "localhost:3000");
379+
expect(resetPostureCache).toHaveBeenCalled();
380+
});
381+
382+
it("stops publishing identity once another process disables telemetry", () => {
383+
canaryDecisions.mockReturnValue({});
384+
readConfig.mockReturnValue({ anonymousId: "id-1", bucketSeed: "seed-1" });
385+
386+
shouldTrack.mockReturnValue(true);
387+
expect(buildStudioHeadScriptsForHost("", "localhost:3000")).toContain("__HF_CLI_DISTINCT_ID");
388+
389+
// `hyperframes telemetry disable` in another terminal.
390+
shouldTrack.mockReturnValue(false);
391+
const after = buildStudioHeadScriptsForHost("", "localhost:3000");
392+
expect(after).not.toContain("__HF_CLI_DISTINCT_ID");
393+
expect(after).not.toContain("__HF_CLI_BUCKET_SEED");
394+
});
395+
});
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
import { describe, expect, it, vi, beforeEach } from "vitest";
2+
3+
// The suppression half of the cross-process opt-out. `hyperframes preview` is
4+
// long-lived, and Studio has no poller for /api/telemetry-identity, so a
5+
// render that finishes AFTER `hyperframes telemetry disable` ran in another
6+
// terminal used to report its outcome anyway: shouldTrack() had cached `true`
7+
// at boot and nothing ever asked again.
8+
//
9+
// This pins the mechanism the render-outcome path depends on — refresh, then
10+
// emit — at the layer where the event is actually dropped.
11+
12+
vi.stubEnv("HYPERFRAMES_NO_TELEMETRY", "");
13+
vi.stubEnv("DO_NOT_TRACK", "");
14+
15+
const configState = { telemetryEnabled: true };
16+
vi.mock("./config.js", () => ({
17+
readConfig: () => ({ anonymousId: "anon-1", telemetryEnabled: configState.telemetryEnabled }),
18+
writeConfig: () => {},
19+
}));
20+
vi.mock("../utils/env.js", () => ({ isDevMode: () => false }));
21+
vi.mock("./canary.js", () => ({ canaryEventProperties: () => ({}) }));
22+
23+
const enqueue = vi.fn();
24+
vi.mock("./transport.js", () => ({
25+
enqueue: (...args: unknown[]) => enqueue(...args),
26+
flush: vi.fn(),
27+
flushSync: vi.fn(),
28+
}));
29+
30+
const { trackEvent, resetTelemetryPostureCache, shouldTrack } = await import("./client.js");
31+
32+
beforeEach(() => {
33+
configState.telemetryEnabled = true;
34+
enqueue.mockClear();
35+
resetTelemetryPostureCache();
36+
});
37+
38+
describe("telemetry posture refresh", () => {
39+
it("stops emitting once another process disables telemetry", () => {
40+
trackEvent("render_complete", {});
41+
expect(enqueue).toHaveBeenCalledTimes(1);
42+
43+
// `hyperframes telemetry disable` elsewhere, mid-render.
44+
configState.telemetryEnabled = false;
45+
resetTelemetryPostureCache();
46+
47+
trackEvent("render_complete", {});
48+
expect(enqueue, "outcome emitted after the user opted out").toHaveBeenCalledTimes(1);
49+
});
50+
51+
// The memo is load-bearing for a CLI command — one process, one answer, asked
52+
// once per event. Dropping it entirely would be a per-event config read.
53+
it("still caches within a posture, so it is not a per-event disk read", () => {
54+
expect(shouldTrack()).toBe(true);
55+
configState.telemetryEnabled = false;
56+
expect(shouldTrack(), "changed without an explicit refresh").toBe(true);
57+
resetTelemetryPostureCache();
58+
expect(shouldTrack()).toBe(false);
59+
});
60+
61+
it("re-enables after the user opts back in", () => {
62+
configState.telemetryEnabled = false;
63+
resetTelemetryPostureCache();
64+
trackEvent("render_complete", {});
65+
expect(enqueue).not.toHaveBeenCalled();
66+
67+
configState.telemetryEnabled = true;
68+
resetTelemetryPostureCache();
69+
trackEvent("render_complete", {});
70+
expect(enqueue).toHaveBeenCalledTimes(1);
71+
});
72+
});

‎packages/cli/src/telemetry/config.test.ts‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -276,6 +276,27 @@ describe("install-state rollover (breaker survives a config re-mint)", () => {
276276
expect(readConfigFresh().predecessorFound).toBeUndefined();
277277
});
278278

279+
// A full reset in a LONG-LIVED process. The memo that says "this process
280+
// already mirrored the state file" stayed set after the file was deleted, so
281+
// the mirror was never recreated: the freshly minted seed lived in
282+
// config.json alone, and the NEXT config-only re-mint rolled a THIRD seed
283+
// instead of inheriting the second. The old test stopped at the second mint
284+
// and so missed the durability half entirely.
285+
it("recreates install-state after a full wipe, so the new seed's lineage is durable", () => {
286+
const first = readConfig().bucketSeed;
287+
fsState.files.delete(CONFIG_PATH);
288+
fsState.files.delete(STATE_PATH);
289+
290+
const second = readConfigFresh().bucketSeed;
291+
expect(second, "a full reset must mint a new cohort").not.toBe(first);
292+
expect(fsState.files.has(STATE_PATH), "state file must be recreated").toBe(true);
293+
expect(stateFile()["bucketSeed"]).toBe(second);
294+
295+
// Config-only re-mint: the seed must now be inherited, not rolled again.
296+
fsState.files.delete(CONFIG_PATH);
297+
expect(readConfigFresh().bucketSeed, "lineage lost after reset").toBe(second);
298+
});
299+
279300
// The move: state used to live in ~/.local/state/hyperframes/ so it would
280301
// survive `rm -rf ~/.hyperframes`. Review rejected persisting state outside
281302
// the config dir to defeat the user's reset, so it now shares CONFIG_DIR.

‎packages/cli/src/telemetry/config.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -317,6 +317,16 @@ function applyInstallState(config: HyperframesConfig, wantFired: boolean): void
317317

318318
function syncInstallState(config: HyperframesConfig): boolean {
319319
const wantFired = config.deParallelRouterTrialFired === true;
320+
// The memo says "this process already wrote the state file". That is only
321+
// true while the file is still there. `rm -rf ~/.hyperframes` under a
322+
// long-lived preview left the memo set, so the mirror was never recreated:
323+
// the freshly minted seed lived in config.json alone, and the NEXT
324+
// config-only re-mint rolled a third seed instead of inheriting the second.
325+
// One existsSync on a path we are about to write anyway.
326+
if (stateMarkerSynced && !existsSync(STATE_FILE)) {
327+
stateMarkerSynced = false;
328+
stateFiredSynced = false;
329+
}
320330
if (stateMarkerSynced && (stateFiredSynced || !wantFired)) return true;
321331
try {
322332
applyInstallState(config, wantFired);

‎packages/core/src/canaryRegistry.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,9 @@ export interface CanaryDefinition {
3838
/**
3939
* ISO date after which this canary is overdue for removal. A canary that
4040
* outlives its rollout is a permanent fork of the product with none of the
41-
* review a permanent fork would have received. `assertNoOverdueCanaries`
42-
* turns the date into a failing test rather than a good intention.
41+
* review a permanent fork would have received. The scheduled `Canary sunset`
42+
* workflow runs `scripts/check-canary-sunset.ts` weekly and fails once the
43+
* date passes, so this is an enforced deadline rather than a good intention.
4344
*/
4445
sunsetAfter: string;
4546
}

0 commit comments

Comments
 (0)