Skip to content

test(pr-to-video): pin the code-vocabulary section of the frame packet - #2900

Merged
jrusso1020 merged 1 commit into
mainfrom
test/pr-to-video-frame-packets
Jul 30, 2026
Merged

test(pr-to-video): pin the code-vocabulary section of the frame packet#2900
jrusso1020 merged 1 commit into
mainfrom
test/pr-to-video-frame-packets

Conversation

@jrusso1020

Copy link
Copy Markdown
Collaborator

What

Three test cases in skills/pr-to-video/scripts/workflow-guardrails.test.mjs covering the code-vocabulary half of this skill's frame-packet delta, plus the skills-manifest.json regeneration that goes with a changed skill.

Why

skills/pr-to-video/scripts/frame-packets.mjs is a thin wrapper over the shared packet builder, and its header names the two reasons it exists:

this file pins this workflow's paths plus its two behavioral differences: a code frame must carry an upstream-selected ### Source excerpt, and code frames get a code-vocabulary excerpt appended to their packet.

The first difference is pinned — "code frames without an upstream-selected excerpt fail before dispatch" covers it. The second had no test anywhere in the repo. I confirmed that rather than assuming it: replacing the body of codeVocabularySection with return "", which deletes the feature outright, left all 455 skills tests green.

That matters because the appended section is what tells a frame worker which registry block to install and what its inputs are. The worker never opens code-vocabulary.md itself, so if the section silently stopped being emitted, the symptom would be workers hand-building code motion instead of installing the purpose-built block — a quality regression with no red check anywhere.

One existing assertion touches this code path (assert.doesNotMatch(codePacket, /code-scroll/) on the #1092 packet test), but it pins the narrowing only. It passes trivially when the section is empty, so it cannot detect deletion.

How

The cases live beside the existing frame-packets tests in workflow-guardrails.test.mjs rather than in a new frame-packets.test.mjs, so this skill keeps one test home per module. (Both sibling skills use the frame-packets.test.mjs name, but their wrappers carry no delta — this skill put its packet tests in the guardrails file first, and splitting them across two files now would be worse than the naming asymmetry.)

  • a code frame carries the vocabulary excerpt for the block it names — a focal: code-diff frame's packet gets the ## Code block excerpt (code-diff) section and the block's own vocabulary line.
  • a code block the vocabulary does not describe still names itself for install — an unrecognized code-* id falls back to Use registry block \`.` rather than emitting nothing, so the worker still learns what to install.
  • a mechanism frame gets no code-block section at all — a non-code focal appends nothing.

Assertions are on the section's structure and the block id, not on the vocabulary's prose, so rewording code-vocabulary.md will not break them.

The code-vocabulary.md-missing branch inside codeVocabularySection is deliberately not covered: SKILL_DIR is resolved from import.meta.url, so that branch is only reachable by deleting a checked-in file.

Test plan

  • node --test on the full CI-discovered skills set: 458 pass, 0 fail (455 before, +3 here).

  • Sensitivity checked, not just presence: with codeVocabularySection neutered to return "", the two positive cases fail and the seven pre-existing tests in the file stay green. Reverted before committing.

  • bun scripts/lint-skills.ts clean (31 skill files), node scripts/check-skill-mirror.mjs clean (24 files byte-identical).

  • bun packages/cli/scripts/gen-skills-manifest.ts --check reported pr-to-video out of date, regenerated, re-checked in sync (19 skills).

  • oxfmt --check clean on the edited file.

  • Unit tests added/updated

  • Manual testing performed

  • Documentation updated (if applicable)

— Rames Jusso

`frame-packets.mjs` declares two behavioral differences from the shared
packet builder: a code frame must carry an upstream-selected
`### Source excerpt`, and code frames get a code-vocabulary excerpt
appended to their packet.

The first is pinned by "code frames without an upstream-selected excerpt
fail before dispatch". The second was not pinned at all. Replacing
`codeVocabularySection` with `return ""` left all 455 skills tests green,
so the section a frame worker reads to pick and install its registry
block could have been dropped with nothing going red.

Adds three cases beside the existing frame-packets tests: the excerpt is
emitted for a named block, a block the vocabulary does not describe still
names itself for install, and a mechanism frame gets no code-block
section at all. The first two fail against the neutered version; the
third holds the negative case.

Regenerates skills-manifest.json for the changed skill.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Substantive notes for the approval above:

Focused review of both changed files and the owning codeVocabularySection production path.

Strengths

  • skills/pr-to-video/scripts/workflow-guardrails.test.mjs:148 pins the named-block excerpt boundary and block identity without coupling to mutable vocabulary prose.
  • skills/pr-to-video/scripts/workflow-guardrails.test.mjs:163 separately pins the unknown-code-* fallback instruction, while :179 proves non-code mechanism frames receive no code-block section.
  • skills-manifest.json:61 is the only generated delta, and the manifest, skill lint, and mirror checks all reproduce cleanly.

Independent verification

  • Focused file: 11/11 pass; full discovered skill suite: 168 pass, 0 fail.
  • Mutation check: replacing the complete helper body with return "" makes exactly the two positive pins fail while the mechanism negative remains green.
  • The untested missing-code-vocabulary.md precondition is acceptable: its observable output is the same fallback pinned by the unknown-block case, and the checked-in reference is protected by manifest/package validation.
  • Exact head 56c24ba543686ab254799e48a2dd9d3a0b227c17 is terminal green; BLOCKED was reviewer-gate only.

Verdict: APPROVE
Reasoning: The tests close a measured regression gap, are mutation-sensitive, avoid brittle prose assertions, and introduce no production behavior change.

— Magi

@jrusso1020
jrusso1020 merged commit 7b3d3db into main Jul 30, 2026
42 checks passed
@jrusso1020
jrusso1020 deleted the test/pr-to-video-frame-packets branch July 30, 2026 18:40
dahans-msft2 pushed a commit to dahans-msft2/hyperframes that referenced this pull request Aug 6, 2026
heygen-com#2900)

Adds three test cases pinning the code-vocabulary section that frame-packets.mjs appends to code frames. Deleting codeVocabularySection outright left all 455 skills tests green before this change; the only assertion touching it was a doesNotMatch that passes trivially when the section is empty.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants