fix(studio): keep dense keyframes readable (#2925)

* perf(studio): define timeline viewport budgets and fixtures

* test(studio): gate timeline viewport performance in Chromium

* refactor(studio): isolate clip drag lifecycle

* refactor(studio): extract timeline render contracts

* perf(studio): centralize timeline viewport geometry

* perf(studio): follow playhead across virtualized rows

* perf(studio): add timeline clip-window index primitive

* perf(studio): virtualize timeline clip windows

* perf(studio): stop timeline scroll work when row virtualization is off

The row virtualization stack made the timeline publish a viewport snapshot
on every scroll frame and swap `renderClipContent` across every mounted clip
at gesture start and settle. Both are windowing concessions, and neither was
gated on the flag, so the build users actually run paid for them while
mounting all 1,000 clips anyway. Measured on a 3,000-clip project: median
scroll step 16.6ms to 76.9ms, p95 17.9ms to 189.4ms, 40 long tasks to 247.

Gate both on the row virtualization flag. The scroll path now stops at the
door when the flag is off, so `isScrolling` stays false and resize-driven
and programmatic syncs still publish through the immediate path.

The flag moves into its own module: the scroll-viewport hook needs to read
it, and the virtualization hook already imports the viewport snapshot type
back, which would have closed an import cycle.

Also release the perf fixture lease from the fixture rather than from the
test-hook effect. Loading a fixture writes player state, which changed that
effect's dependency identities and tore it down on the next frame, so the
lease was revoked moments after it was taken and live iframe discovery
overwrote the fixture before the gate could measure it.

The e2e gate gains a flag-off arm (`test:timeline-default`, 1,000 elements)
next to the existing flag-on one. It refuses the 50,000-element combination,
verifies from the mounted DOM that the server under test matches the
requested flag, and skips the DOM-size budgets for the unvirtualized build
rather than relaxing them, so a skipped budget never reads as a passed one.

Verified against a live Studio dev server on the fixture project:

  flag off, before: interactionP95 303.1ms, longest task 194ms, 0/5 runs pass
  flag off, after:  interactionP95  33.6ms, longest task   0ms, 5/5 runs pass
  flag on,  after:  interactionP95  33.2ms, 4/5 runs pass, exit 0

The flag-on arm's fourth run reproducibly reports a 55-58ms long task
against a 50ms budget. That is the residual tail of the window swap itself,
tracked separately and not addressed here.

* ci(studio): run the timeline viewport gate on studio changes

The gate has existed since the row virtualization stack landed but nothing
under `.github/` referenced it, so it only ever ran when someone ran it by
hand. That is how the flag-off scroll regression reached eight merged-ready
PRs without anything noticing.

Adds a `studio-timeline-viewport` job that boots two Studio dev servers, one
per flag state, and runs both arms of the gate against them. Two servers are
needed because row virtualization is read from `import.meta.env` at module
load, so one process cannot serve both builds.

Scoped to a new `studio` paths filter rather than the broad `code` one: the
gate only says anything about `packages/studio`, `packages/core` and
`packages/studio-server`.

Adds a `ci` tier. It applies the constrained budgets without any emulation,
because a hosted runner is already slower and noisier than the machine the
strict numbers were recorded on, while the existing `low-resource` tier would
throttle it a further 4x and measure the throttle rather than the build.

The fixture composition is tracked under `tests/e2e/fixtures` but Studio
resolves projects from the gitignored `data/projects`, so the job copies it
into place instead of a project directory being committed.

Both arms run in about 7 seconds each locally, so the job cost is almost
entirely dependency install and the workspace build it shares with
`studio-load-smoke`.

* fix(ci): preserve both timeline gate evidence arms

* ci(studio): report timeline gate arm statuses

* ci(studio): require timeline gate evidence artifacts

* fix(studio): keep dense keyframes readable

* fix(ci): resolve timeline stack audit findings

Author: miguel-heygen

.fallowrc.jsonc

@@ -631,6 +631,11 @@
     // complexity pre-dates the computed-timeline work. Exempted at file level
     // rather than refactored as scope creep.
     "ignore": [
+      // useGestureRecording.ts: readBasePosition/connectGsapRuntime/tick are
+      // inherited gesture-runtime control flow. This stack only changes
+      // recordSample to coalesce display-rate events onto authored frames;
+      // the import and helper insertion shift the untouched functions' lines.
+      "packages/studio/src/hooks/useGestureRecording.ts",
       // sourcePatcher.ts: resolveSourceFile / splitInlineStyleDeclarations /
       // patch*InTag pre-date this PR; only the PatchOperation type gained two
       // optional fields, but the line-shift fingerprint re-flags the inherited

.github/workflows/ci.yml

@@ -36,6 +36,7 @@ jobs:
       skills: ${{ steps.filter.outputs.skills }}
       codex_plugin: ${{ steps.filter.outputs.codex_plugin }}
       gcp_beginframe: ${{ steps.filter.outputs.gcp_beginframe }}
+      studio: ${{ steps.filter.outputs.studio }}
     steps:
       # Force git-based change detection instead of the pull_request REST API.
       # The API path can fail the whole workflow on transient listFiles
@@ -77,6 +78,12 @@ jobs:
               - "scripts/package-codex-plugin.mjs"
               - "package.json"
               - ".github/workflows/ci.yml"
+            studio:
+              - "packages/studio/**"
+              - "packages/core/**"
+              - "packages/studio-server/**"
+              - "bun.lock"
+              - ".github/workflows/ci.yml"
             gcp_beginframe:
               - "packages/gcp-cloud-run/Dockerfile"
               - "packages/aws-lambda/scripts/probe-beginframe.ts"
@@ -496,6 +503,97 @@ jobs:
 
           kill $SERVER_PID 2>/dev/null || true
 
+  studio-timeline-viewport:
+    name: "Studio: timeline viewport gate"
+    needs: [changes]
+    if: needs.changes.outputs.studio == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 12
+    steps:
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
+      - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
+      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
+        with:
+          node-version: 22
+      - run: bun install --frozen-lockfile
+      # Same reason as studio-load-smoke: vite.config.ts is loaded by Node and
+      # resolves the workspace packages through their "node" export condition.
+      - run: bun run --filter '@hyperframes/{parsers,lint,studio-server}' build
+      - run: bun run --cwd packages/core build
+      - run: bun run --cwd packages/core build:hyperframes-runtime
+      - name: Install the fixture as a Studio project
+        # Studio resolves projects from packages/studio/data/projects, which is
+        # gitignored. The fixture composition is tracked under tests/e2e, so
+        # copy it into place rather than committing a project directory.
+        run: |
+          mkdir -p packages/studio/data/projects
+          cp -R packages/studio/tests/e2e/fixtures/timeline-virtualization \
+            packages/studio/data/projects/timeline-virtualization
+      - name: Run both arms of the timeline viewport gate
+        run: |
+          set -euo pipefail
+
+          # Two servers, because row virtualization is read from import.meta.env
+          # at module load: one process cannot serve both builds.
+          bun run --cwd packages/studio dev -- --port 5313 --strictPort &
+          DEFAULT_PID=$!
+          VITE_STUDIO_TIMELINE_ROW_VIRTUALIZATION_ENABLED=1 \
+            bun run --cwd packages/studio dev -- --port 5314 --strictPort &
+          VIRTUALIZED_PID=$!
+          trap 'kill $DEFAULT_PID $VIRTUALIZED_PID 2>/dev/null || true' EXIT
+
+          for i in $(seq 1 60); do
+            if curl -sf http://localhost:5313/ >/dev/null 2>&1 \
+              && curl -sf http://localhost:5314/ >/dev/null 2>&1; then break; fi
+            sleep 1
+          done
+          if ! curl -sf http://localhost:5313/ >/dev/null 2>&1 \
+            || ! curl -sf http://localhost:5314/ >/dev/null 2>&1; then
+            echo "FAIL: studio dev servers did not start"
+            exit 1
+          fi
+
+          # The default build first. It is the one users get, and the arm that
+          # caught the regression this gate exists for.
+          # Capture both statuses so either failure still leaves two evidence files.
+          DEFAULT_STATUS=0
+          STUDIO_URL="http://localhost:5313/#project/timeline-virtualization" \
+            TIMELINE_ROW_VIRTUALIZATION=off \
+            TIMELINE_ELEMENT_COUNT=1000 \
+            TIMELINE_TIER=ci \
+            node packages/studio/tests/e2e/timeline-virtualization.mjs \
+            | tee /tmp/timeline-gate-default.json \
+            || DEFAULT_STATUS=$?
+
+          VIRTUALIZED_STATUS=0
+          STUDIO_URL="http://localhost:5314/#project/timeline-virtualization" \
+            TIMELINE_ROW_VIRTUALIZATION=on \
+            TIMELINE_ELEMENT_COUNT=50000 \
+            TIMELINE_TIER=ci \
+            node packages/studio/tests/e2e/timeline-virtualization.mjs \
+            | tee /tmp/timeline-gate-virtualized.json \
+            || VIRTUALIZED_STATUS=$?
+
+          {
+            echo "### Timeline viewport gate"
+            echo "- Default arm exit: ${DEFAULT_STATUS}"
+            echo "- Virtualized arm exit: ${VIRTUALIZED_STATUS}"
+          } >> "$GITHUB_STEP_SUMMARY"
+
+          if (( DEFAULT_STATUS != 0 || VIRTUALIZED_STATUS != 0 )); then
+            echo "FAIL: default=${DEFAULT_STATUS}, virtualized=${VIRTUALIZED_STATUS}"
+            exit 1
+          fi
+      - name: Upload gate evidence
+        # The gate's whole output is machine-readable evidence, and a red run is
+        # exactly when someone needs to read it. Keep it on failure too.
+        if: always()
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+        with:
+          name: timeline-viewport-gate-evidence
+          path: /tmp/timeline-gate-*.json
+          if-no-files-found: error
+
   smoke-global-install:
     name: "Smoke: global install"
     needs: [changes, build]

bun.lock

@@ -49,8 +49,10 @@
     "build": "vite build && tsup",
     "typecheck": "tsc --noEmit",
     "test": "vitest run",
+    "test:timeline-virtualization": "TIMELINE_ROW_VIRTUALIZATION=on TIMELINE_ELEMENT_COUNT=50000 node tests/e2e/timeline-virtualization.mjs",
     "test:watch": "vitest",
-    "report:sdk-cutover": "bun src/utils/sdkCutoverPolicy.report.ts"
+    "report:sdk-cutover": "bun src/utils/sdkCutoverPolicy.report.ts",
+    "test:timeline-default": "TIMELINE_ROW_VIRTUALIZATION=off TIMELINE_ELEMENT_COUNT=1000 node tests/e2e/timeline-virtualization.mjs"
   },
   "dependencies": {
     "@codemirror/autocomplete": "^6.20.1",
@@ -70,6 +72,7 @@
     "@hyperframes/sdk": "workspace:*",
     "@hyperframes/studio-server": "workspace:*",
     "@phosphor-icons/react": "^2.1.10",
+    "@tanstack/react-virtual": "^3.14.6",
     "bpm-detective": "^2.0.5",
     "dompurify": "^3.2.4",
     "gsap": "^3.13.0",

packages/studio/package.json

@@ -118,8 +118,13 @@ export function TimelineToolbar({ domEditSession, onSplitElement }: TimelineTool
   // the selection changes, not only on the next playhead tick.
   const selectedElementId = usePlayerStore((s) => s.selectedElementId);
   const elements = usePlayerStore((s) => s.elements);
+  const timelineFitPps = usePlayerStore((s) => s.timelineFitPps);
   const { zoomMode, manualZoomPercent, setZoomMode, setManualZoomPercent } = useTimelineZoom();
-  const displayedTimelineZoomPercent = getTimelineZoomPercent(zoomMode, manualZoomPercent);
+  const displayedTimelineZoomPercent = getTimelineZoomPercent(
+    zoomMode,
+    manualZoomPercent,
+    timelineFitPps,
+  );
   const {
     state: keyframeState,
     isMotionPath: keyframeIsMotionPath,
@@ -418,7 +423,7 @@ export function TimelineToolbar({ domEditSession, onSplitElement }: TimelineTool
               onClick={() => {
                 setZoomMode("manual");
                 setManualZoomPercent(
-                  getNextTimelineZoomPercent("out", zoomMode, manualZoomPercent),
+                  getNextTimelineZoomPercent("out", zoomMode, manualZoomPercent, timelineFitPps),
                 );
               }}
               className={flatIdle}
@@ -430,12 +435,14 @@ export function TimelineToolbar({ domEditSession, onSplitElement }: TimelineTool
             type="range"
             min="0"
             max="100"
-            value={timelineZoomPercentToSlider(displayedTimelineZoomPercent)}
+            value={timelineZoomPercentToSlider(displayedTimelineZoomPercent, timelineFitPps)}
             title={`${displayedTimelineZoomPercent}%`}
             aria-label="Timeline zoom"
             onChange={(e) => {
               setZoomMode("manual");
-              setManualZoomPercent(timelineSliderToZoomPercent(Number(e.target.value)));
+              setManualZoomPercent(
+                timelineSliderToZoomPercent(Number(e.target.value), timelineFitPps),
+              );
             }}
             // h-6 on the input is the 24x24 WCAG 2.2 (2.5.8) target: the visible
             // track stays 2px and the thumb 10px, only the pointer box grows.
@@ -447,7 +454,9 @@ export function TimelineToolbar({ domEditSession, onSplitElement }: TimelineTool
               aria-label="Zoom in"
               onClick={() => {
                 setZoomMode("manual");
-                setManualZoomPercent(getNextTimelineZoomPercent("in", zoomMode, manualZoomPercent));
+                setManualZoomPercent(
+                  getNextTimelineZoomPercent("in", zoomMode, manualZoomPercent, timelineFitPps),
+                );
               }}
               className={flatIdle}
             >

packages/studio/src/components/TimelineToolbar.tsx

@@ -48,6 +48,7 @@ export interface NLEContextValue {
   compositionLoading: boolean;
   setCompositionLoading: (loading: boolean) => void;
   timelineDisabled: boolean;
+  timelineSessionEpoch: number;
   hasLoadedOnceRef: React.MutableRefObject<boolean>;
   // preview composition size (for preview block drop)
   previewCompositionSize: { width: number; height: number } | null;
@@ -103,7 +104,7 @@ export function NLEProvider({
   // project would otherwise keep rendering (and re-fetching from) the old project
   // after switching.
   useEffect(() => {
-    usePlayerStore.getState().reset();
+    usePlayerStore.getState().beginTimelineSession(projectId);
     useAssetPreviewStore.getState().clearPreviewAsset();
   }, [projectId]);
 
@@ -289,6 +290,7 @@ export function NLEProvider({
     setCompositionLoadingRaw(loading);
   }, []);
   const timelineDisabled = shouldDisableTimelineWhileCompositionLoading(compositionLoading);
+  const timelineSessionEpoch = usePlayerStore((state) => state.timelineSessionEpoch);
 
   useEffect(() => {
     onCompositionLoadingChange?.(compositionLoading);
@@ -319,6 +321,7 @@ export function NLEProvider({
     compositionLoading,
     setCompositionLoading,
     timelineDisabled,
+    timelineSessionEpoch,
     hasLoadedOnceRef,
     previewCompositionSize,
     setPreviewCompositionSize,

packages/studio/src/components/nle/NLEContext.tsx

@@ -127,6 +127,7 @@ export function TimelinePane({
     persistTimelineH,
     containerRef,
     timelineDisabled,
+    timelineSessionEpoch,
   } = useNLEContext();
 
   // Move/resize/split come from the timeline edit context, not props — the
@@ -271,6 +272,7 @@ export function TimelinePane({
         >
           <div className="flex-shrink-0">{timelineToolbar}</div>
           <Timeline
+            sessionEpoch={timelineSessionEpoch}
             onSeek={seek}
             onDrillDown={handleDrillDown}
             renderClipContent={renderClipContent}

packages/studio/src/components/nle/TimelinePane.tsx

@@ -3,7 +3,7 @@
 import React, { act } from "react";
 import { afterEach, describe, expect, it, vi } from "vitest";
 import { mountReactHarness } from "./domSelectionTestHarness";
-import { useGestureRecording } from "./useGestureRecording";
+import { useGestureRecording, type GestureSample } from "./useGestureRecording";
 
 Reflect.set(globalThis, "IS_REACT_ACT_ENVIRONMENT", true);
 
@@ -13,6 +13,58 @@ afterEach(() => {
 });
 
 describe("useGestureRecording", () => {
+  it("keeps only the latest gesture sample in each output frame", () => {
+    const animationFrames: FrameRequestCallback[] = [];
+    vi.stubGlobal(
+      "requestAnimationFrame",
+      vi.fn((callback: FrameRequestCallback) => {
+        animationFrames.push(callback);
+        return animationFrames.length;
+      }),
+    );
+    vi.stubGlobal("cancelAnimationFrame", vi.fn());
+    let nowMs = 0;
+    vi.spyOn(performance, "now").mockImplementation(() => nowMs);
+
+    const iframe = document.createElement("iframe");
+    document.body.append(iframe);
+    const element = iframe.contentDocument!.createElement("div");
+    element.id = "card";
+    iframe.contentDocument!.body.append(element);
+
+    let recording: ReturnType<typeof useGestureRecording> | null = null;
+    function Harness() {
+      recording = useGestureRecording();
+      return null;
+    }
+    const root = mountReactHarness(<Harness />);
+
+    act(() => recording?.startRecording(element, iframe));
+    document.dispatchEvent(new MouseEvent("pointermove", { bubbles: true, clientX: 10 }));
+    nowMs = 1;
+    act(() => animationFrames.shift()?.(1));
+    document.dispatchEvent(new MouseEvent("pointermove", { bubbles: true, clientX: 20 }));
+    nowMs = 10;
+    act(() => animationFrames.shift()?.(10));
+    document.dispatchEvent(new MouseEvent("pointermove", { bubbles: true, clientX: 40 }));
+    nowMs = 20;
+    act(() => animationFrames.shift()?.(20));
+    document.dispatchEvent(new MouseEvent("pointermove", { bubbles: true, clientX: 50 }));
+    nowMs = 30;
+    act(() => animationFrames.shift()?.(30));
+
+    let samples: GestureSample[] = [];
+    act(() => {
+      samples = recording?.stopRecording() ?? [];
+    });
+
+    expect(samples).toEqual([
+      { time: 0, properties: { x: 10, y: 0 } },
+      { time: 1 / 30, properties: { x: 40, y: 0 } },
+    ]);
+    act(() => root.unmount());
+  });
+
   it("releases the preview runtime when unmounted during a recording", () => {
     const cancelAnimationFrame = vi.fn();
     vi.stubGlobal(

packages/studio/src/hooks/useGestureRecording.test.tsx

@@ -1,5 +1,6 @@
 import { useCallback, useEffect, useRef, useState } from "react";
 import { usePlayerStore, liveTime } from "../player/store/playerStore";
+import { frameToSeconds, secondsToFrame } from "../player/lib/time";
 
 export interface GestureSample {
   time: number;
@@ -139,7 +140,18 @@ function recordSample(r: RecordingRefs, time: number, properties: Record<string,
   // stripStudioEditsFromTarget in studio-api), so the keyframes must already
   // include it. Subtracting it made the committed gesture play shoved off by the
   // offset (the offset was removed twice).
-  r.samples.push({ time, properties: { ...properties } });
+  const frame = secondsToFrame(time);
+  const sample = { time: frameToSeconds(frame), properties: { ...properties } };
+  const lastIndex = r.samples.length - 1;
+  const lastSample = r.samples[lastIndex];
+  // Gesture events follow the display refresh rate (often 60/120Hz), but the
+  // Studio preview is authored at 30fps. Keep the latest pointer state for each
+  // output frame so recording cannot create visually indistinguishable piles.
+  if (lastSample?.time === sample.time) {
+    r.samples[lastIndex] = sample;
+  } else {
+    r.samples.push(sample);
+  }
   r.trail.push({ x: r.pointer.x, y: r.pointer.y });
 }