fix(cli): ignore generated caches in project watcher

[miguel-heygen]

Summary

• ignore .transcode-cache, .thumbnails, and .waveform-cache in the recursive project watcher
• exclude those generated caches from the Storyboard project signature
• omit transcode and waveform caches from GET /projects/:id file listings (.thumbnails was already omitted)
• retain positive behavior for real project edits
• cover watcher, signature, and listing boundaries with focused regressions

Fixes #2951

Root cause and scope

Generated cache paths crossed multiple project-change boundaries. Watcher events could become Studio file-change invalidations, and generated cache writes also changed the signature polled by Storyboard every two seconds. The project listing additionally exposed transcode and waveform cache files.

Static waveform proof:

• packages/studio-server/src/routes/waveform.ts:18-39 creates and writes .waveform-cache
• packages/studio/src/hooks/useRenderClipContent.ts:63-69 builds the timeline waveform route
• packages/studio/src/player/components/AudioWaveform.tsx:94-100 fetches it

routes/files.ts is a separate rename-reference scanner and remains unchanged because no concrete corruption RED was established. No shared generated-directory constant was added because watcher, signature, and listing policies are distinct contracts across package boundaries.

TDD evidence

Cycle 1: proxy and thumbnail watcher caches

RED before production:

vitest run packages/cli/src/server/fileWatcher.test.ts
1 failed | 2 passed
.transcode-cache/proxy.mp4: expected true to be false

The investigation independently observed real watcher callbacks after 301 ms for both .transcode-cache/proxy.mp4 and .thumbnails/frame.jpg mtime touches.

GREEN after adding only those exclusions: 3/3 focused watcher tests passed.

Cycle 2: waveform watcher cache

At the committed cycle-1 production state, the test-only assertion produced RED:

1 failed | 2 passed
.waveform-cache/peaks.json: expected true to be false

GREEN after adding only .waveform-cache: 3/3 focused watcher tests passed.

Real watcher control:

{"transcodeEvents":[],"thumbnailEvents":[],"waveformEvents":[],"sourceEvents":["src/scene.ts"]}

Cycle 3: test naming review

Renamed the watcher test to skips generated and dependency directories; no production behavior changed. Focused watcher tests remained 3/3 GREEN.

Cycle 4: signature and project listing

Route-level tests were added first on commit 3eb7b1ffd.

RED before production edits:

projects.test.ts: 2 failed | 5 passed
signature: all three generated cache writes changed /projects/:id/signature
listing: .transcode-cache/proxy.mp4 was present in GET /projects/:id files

The signature test collected distinct signatures after each generated write, while the listing test exercised the actual route boundary. Only after that RED:

• .thumbnails, .transcode-cache, and .waveform-cache were added to SIGNATURE_EXCLUDED_DIRS
• .transcode-cache and .waveform-cache were added to the safePath.ts file-tree ignore set
• .waveform-cache was moved after .vite in the watcher list (ordering only)

GREEN:

packages/studio-server/src/routes/projects.test.ts: 7/7 passed

Independent real-filesystem signature control:

{
  "initial":"1af301bcc1150fd8a80f883d",
  "generatedSignatures":{
    ".transcode-cache":"1af301bcc1150fd8a80f883d",
    ".thumbnails":"1af301bcc1150fd8a80f883d",
    ".waveform-cache":"1af301bcc1150fd8a80f883d"
  },
  "sourceSignature":"c335cd75a57722379cf40281"
}

All generated writes preserve the signature; a real src/scene.ts write changes it.

Verification

• focused route suite: 7 passed
• nearby studio-server suites: 3 files, 75 tests passed
• nearby CLI server suites: 4 files, 33 tests passed
• oxlint on r4 changed files: 0 warnings, 0 errors
• oxfmt --check on r4 changed files: all correctly formatted
• git diff --check: clean
• r4 commit scope: exactly 4 approved files
• cumulative PR scope: watcher test/implementation plus project signature, safe path, and route test only
• 16 assume-unchanged LFS fixtures remained unstaged
• no dependencies installed

[vanceingalls]

Review — APPROVE

Fix correctly addresses the specific feedback loop identified in #2951: proxy-cache utimesSync → watcher SSE file-change → preview iframe reload → composition re-requests media → repeat. Adding .transcode-cache to WATCHER_EXCLUDED_DIRS breaks the closed loop. The preemptive additions of .thumbnails and .waveform-cache are the same failure mode on adjacent surfaces (waveform generation writes on cache miss; thumbnails are a plausible sibling).

Correctness confirmed.

• shouldWatchProjectFile splits on / and \ and checks each path segment against the Set — nested paths like media/subdir/.transcode-cache/f.mp4 also get excluded. ✓
• Set semantics are order-independent for lookup; alphabetical order in the source is a maintainer signal.
• Test asserts all three new dirs return false via shouldWatchProjectFile.
• Cache dirs verified to exist as described:
  • packages/studio-server/src/helpers/proxyTranscoder.ts writes to .transcode-cache (CACHE_DIR_NAME) and calls utimesSync on hits.
  • packages/studio-server/src/routes/waveform.ts writes .waveform-cache.
  • .thumbnails already excluded in safePath.ts::IGNORE_DIRS, consistent naming.

Non-blocking notes.

1. Alphabetical placement of .waveform-cache. Inserted between .transcode-cache and .vite; alphabetically .waveform-cache (w-a) should come after .vite (v). Purely cosmetic — a Set doesn't care — but the surrounding list is alphabetized, so a future reader will bounce off it. One-line reorder if you want to keep the invariant tight.

2. Deferred parity risk — second-order UX. PR body correctly flags that projectSignature.ts::SIGNATURE_EXCLUDED_DIRS and safePath.ts::IGNORE_DIRS still omit these dirs, deferred pending dedicated tests. Verified the current state:

   • SIGNATURE_EXCLUDED_DIRS: .cache, .git, .hyperframes, .next, .vite, build, coverage, dist, node_modules, outputs, renders — missing all three.
   • safePath.ts::IGNORE_DIRS: .thumbnails, node_modules, .git — missing .transcode-cache + .waveform-cache.

   Practical implication: useProjectSignaturePoll in StoryboardView polls every 2 s and fires onChange when the project signature shifts. projectSignature.ts fingerprints all files (mtime + size) except SIGNATURE_EXCLUDED_DIRS, so .transcode-cache/*.mp4 mtime touches from markCacheEntryUsed() will still shift the signature — meaning storyboard boards will refetch every 2 s while proxies are actively being served, even after this fix lands.

   That's a distinct symptom from #2951 (storyboard refetch ≠ preview iframe reload loop, and the preview iframe is fixed by this PR), so it's fair to defer. Worth filing a follow-up so it doesn't get lost — the fix is trivial (add the same three entries to SIGNATURE_EXCLUDED_DIRS) but needs its own test scaffolding, which is why the deferral makes sense.

Verification.

• Fixes #2951's identified loop at the reported reproduction path.
• CI test evidence in PR body is credible (TDD RED → GREEN cycles per exclusion).
• No behavioral change for legit project files; only three dot-prefixed generated dirs added.

APPROVE at bf739a4db2807373a0c294224dd4a101c6ed8954.

— Via

[james-russo-rames-d-jusso]

Loop-break fix is correct — this closes issue #2951 cleanly. Watcher gate at packages/cli/src/server/fileWatcher.ts:33 (shouldWatchProjectFile) now short-circuits .transcode-cache/* mtime touches from markCacheEntryUsed (packages/studio-server/src/helpers/proxyTranscoder.ts:262, called at :484), which was the loop's only actual driver — .thumbnails and .waveform-cache don't call utimesSync. Adding them anyway is proactive (proxies, thumbnails, waveforms are all generated-dir contents that churn during preview activity, whether via LRU touch or first-write).

Concerns

• Sibling exclusion lists went un-updated. Three other places carry overlapping cache-dir lists that this PR doesn't touch:

  • SIGNATURE_EXCLUDED_DIRS at packages/studio-server/src/helpers/projectSignature.ts:18 — missing all three new dirs. Consequence: collectProjectSignatureFiles still descends into .transcode-cache/.thumbnails/.waveform-cache, so any signature-driven change detection churns as proxies/thumbnails/waveforms get written or LRU-touched. Whether that matters depends on the downstream consumer, which I didn't trace exhaustively — worth checking, or leaving deliberately. (You renamed the test from "excluded from signatures" to just "excluded from" between force-pushes, so you may have already noticed the two-list asymmetry — flagging anyway for the record.)
  • walkFiles inline check at packages/studio-server/src/routes/files.ts:499-503 — has .thumbnails and .transcode-cache but not .waveform-cache. Same class of gap.
  • IGNORE_DIRS at packages/studio-server/src/helpers/safePath.ts:9 — only .thumbnails, node_modules, .git. isInHiddenOrVendorDir at line 22 catches the other two via startsWith("."), so downstream discovery is safe; walkDir itself just does wasted I/O on cache-dir descents.

  A shared GENERATED_DIRS constant that all four sites imported would remove this class of drift entirely — bigger than a fix-PR scope, but worth an issue.

Nit

• Alphabetical order in the new list: .waveform-cache (.w) is placed before .vite (.v) at packages/cli/src/server/fileWatcher.ts:19. Purely aesthetic — Set doesn't care — but the rest of the list is alphabetical.

Fix itself LGTM.

— Review by Rames D Jusso

[vanceingalls]

R2 — APPROVE at 1d01b9f2c

Both R1 non-blockers addressed cleanly.

R1 Note 1 (alphabetical placement). Fixed — .waveform-cache now sits after .vite in fileWatcher.ts. Same ordering applied to the new entries in projectSignature.ts.

R1 Note 2 (deferred SIGNATURE_EXCLUDED_DIRS + safePath.IGNORE_DIRS parity). Fully closed:

• packages/studio-server/src/helpers/projectSignature.ts::SIGNATURE_EXCLUDED_DIRS — added .thumbnails, .transcode-cache, .waveform-cache. This is the signature-poll surface that would have kept StoryboardView refetching every 2 s while proxies were being served.
• packages/studio-server/src/helpers/safePath.ts::IGNORE_DIRS — added .transcode-cache, .waveform-cache (.thumbnails was already present). walkDir now hides these from the project file tree.
• packages/studio-server/src/routes/projects.test.ts — two new tests at the real route boundary, not just unit level:
  1. GET /projects/:id/signature: writes files to all three generated dirs, asserts signature unchanged across the three writes, then writes src/scene.ts and asserts signature does change. RED→GREEN for the signature-poll churn.
  2. GET /projects/:id: asserts .transcode-cache/proxy.mp4 and .waveform-cache/peaks.json are not in payload.files. Confirms the safePath.walkDir change is reachable from the listing route.

Scope decisions I agree with.

• Leaving routes/files.ts alone: rename-reference scanner, not listing surface. Fine.
• Rejecting a shared constant: fileWatcher.ts lives in packages/cli, the other two live in packages/studio-server. Different package/layer contracts; cross-package coupling for an incidental overlap would be net-negative. Fine.

Adversarial checks.

• Segment-level match invariant preserved in all three exclusion sites — nested paths like media/subdir/.transcode-cache/f.mp4 are excluded because .split(/[\\/]+/).some(p => set.has(p)) fires on the segment, not the full path.
• SIGNATURE_TEXT_EXTENSIONS unchanged; only the dir filter changed. Non-text files still contribute size/mtime to the fingerprint except under the excluded dirs, which is the correct scope.
• No test-only fix: production code changed in both .ts files that carry the deferred parity gap, tests execute through the real routes.

APPROVE at 1d01b9f2cfe44cb7899410e742a9d553c7807e19.

— Via

[james-russo-rames-d-jusso]

Delta 3eb7b1ff → 1d01b9f2c verified at HEAD — all three R1 items resolved.

• .waveform-cache sort order — moved to after .vite at packages/cli/src/server/fileWatcher.ts:19 ✓
• SIGNATURE_EXCLUDED_DIRS — three additions at packages/studio-server/src/helpers/projectSignature.ts:23-26, and the RED→GREEN test at packages/studio-server/src/routes/projects.test.ts:94-121 (ignores generated cache writes while detecting source edits) locks the invariant on both sides: three sequential cache-dir writes leave the signature unchanged, then a real source edit under src/ moves it. Solid.
• safePath.walkDir — .transcode-cache and .waveform-cache added to IGNORE_DIRS at packages/studio-server/src/helpers/safePath.ts:9-15, plus a project-tree test at projects.test.ts:167-181 confirming neither cache file surfaces via /projects/demo.

Verified the routes/files.ts skip: walkFiles at :494 is only consumed by updateReferences at :518, which is the rename-follow scanner ("After a rename, update all references to the old path in project files"), not the file-listing path. Descending into .waveform-cache/peaks.json there would waste I/O but not corrupt anything meaningful — the guard against real corruption is the !content.includes(oldPath) gate at :527 plus the "never bare filenames" comment. Leaving walkFiles alone is defensible.

Shared-constant rejection is also fine — the three lists live in different packages (cli vs studio-server) and encode subtly different contracts (watcher trigger vs signature invalidation vs discovery traversal); coupling them across package boundaries would trade drift for over-constraint. Local + tested is the right pick here.

CI showed one Smoke: global install failure at the time of review (job 91526931643) — that surface is npx-shim smoke, untouched by this PR's directory-list changes; likely pre-existing flake, worth a quick re-run to confirm.

LGTM from my side.

— Review by Rames D Jusso