test(pr-to-video): pin the code-vocabulary section of the frame packet - #2900
Merged
Conversation
`frame-packets.mjs` declares two behavioral differences from the shared packet builder: a code frame must carry an upstream-selected `### Source excerpt`, and code frames get a code-vocabulary excerpt appended to their packet. The first is pinned by "code frames without an upstream-selected excerpt fail before dispatch". The second was not pinned at all. Replacing `codeVocabularySection` with `return ""` left all 455 skills tests green, so the section a frame worker reads to pick and install its registry block could have been dropped with nothing going red. Adds three cases beside the existing frame-packets tests: the excerpt is emitted for a named block, a block the vocabulary does not describe still names itself for install, and a mechanism frame gets no code-block section at all. The first two fail against the neutered version; the third holds the negative case. Regenerates skills-manifest.json for the changed skill. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
miguel-heygen
approved these changes
Jul 30, 2026
miguel-heygen
left a comment
Collaborator
There was a problem hiding this comment.
Substantive notes for the approval above:
Focused review of both changed files and the owning codeVocabularySection production path.
Strengths
skills/pr-to-video/scripts/workflow-guardrails.test.mjs:148pins the named-block excerpt boundary and block identity without coupling to mutable vocabulary prose.skills/pr-to-video/scripts/workflow-guardrails.test.mjs:163separately pins the unknown-code-*fallback instruction, while:179proves non-code mechanism frames receive no code-block section.skills-manifest.json:61is the only generated delta, and the manifest, skill lint, and mirror checks all reproduce cleanly.
Independent verification
- Focused file: 11/11 pass; full discovered skill suite: 168 pass, 0 fail.
- Mutation check: replacing the complete helper body with
return ""makes exactly the two positive pins fail while the mechanism negative remains green. - The untested missing-
code-vocabulary.mdprecondition is acceptable: its observable output is the same fallback pinned by the unknown-block case, and the checked-in reference is protected by manifest/package validation. - Exact head
56c24ba543686ab254799e48a2dd9d3a0b227c17is terminal green;BLOCKEDwas reviewer-gate only.
Verdict: APPROVE
Reasoning: The tests close a measured regression gap, are mutation-sensitive, avoid brittle prose assertions, and introduce no production behavior change.
— Magi
dahans-msft2
pushed a commit
to dahans-msft2/hyperframes
that referenced
this pull request
Aug 6, 2026
heygen-com#2900) Adds three test cases pinning the code-vocabulary section that frame-packets.mjs appends to code frames. Deleting codeVocabularySection outright left all 455 skills tests green before this change; the only assertion touching it was a doesNotMatch that passes trivially when the section is empty.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Three test cases in
skills/pr-to-video/scripts/workflow-guardrails.test.mjscovering the code-vocabulary half of this skill's frame-packet delta, plus theskills-manifest.jsonregeneration that goes with a changed skill.Why
skills/pr-to-video/scripts/frame-packets.mjsis a thin wrapper over the shared packet builder, and its header names the two reasons it exists:The first difference is pinned —
"code frames without an upstream-selected excerpt fail before dispatch"covers it. The second had no test anywhere in the repo. I confirmed that rather than assuming it: replacing the body ofcodeVocabularySectionwithreturn "", which deletes the feature outright, left all 455 skills tests green.That matters because the appended section is what tells a frame worker which registry block to install and what its inputs are. The worker never opens
code-vocabulary.mditself, so if the section silently stopped being emitted, the symptom would be workers hand-building code motion instead of installing the purpose-built block — a quality regression with no red check anywhere.One existing assertion touches this code path (
assert.doesNotMatch(codePacket, /code-scroll/)on the#1092packet test), but it pins the narrowing only. It passes trivially when the section is empty, so it cannot detect deletion.How
The cases live beside the existing
frame-packetstests inworkflow-guardrails.test.mjsrather than in a newframe-packets.test.mjs, so this skill keeps one test home per module. (Both sibling skills use theframe-packets.test.mjsname, but their wrappers carry no delta — this skill put its packet tests in the guardrails file first, and splitting them across two files now would be worse than the naming asymmetry.)a code frame carries the vocabulary excerpt for the block it names— afocal: code-diffframe's packet gets the## Code block excerpt (code-diff)section and the block's own vocabulary line.a code block the vocabulary does not describe still names itself for install— an unrecognizedcode-*id falls back toUse registry block \`.` rather than emitting nothing, so the worker still learns what to install.a mechanism frame gets no code-block section at all— a non-codefocalappends nothing.Assertions are on the section's structure and the block id, not on the vocabulary's prose, so rewording
code-vocabulary.mdwill not break them.The
code-vocabulary.md-missing branch insidecodeVocabularySectionis deliberately not covered:SKILL_DIRis resolved fromimport.meta.url, so that branch is only reachable by deleting a checked-in file.Test plan
node --teston the full CI-discovered skills set: 458 pass, 0 fail (455 before, +3 here).Sensitivity checked, not just presence: with
codeVocabularySectionneutered toreturn "", the two positive cases fail and the seven pre-existing tests in the file stay green. Reverted before committing.bun scripts/lint-skills.tsclean (31 skill files),node scripts/check-skill-mirror.mjsclean (24 files byte-identical).bun packages/cli/scripts/gen-skills-manifest.ts --checkreportedpr-to-videoout of date, regenerated, re-checked in sync (19 skills).oxfmt --checkclean on the edited file.Unit tests added/updated
Manual testing performed
Documentation updated (if applicable)
— Rames Jusso