Land the lecture evaluation system (benchmark plugin 0.3.0: rubric v2, skill wired) - #5

Merged

mmcky merged 25 commits into

mainfrom

land-evaluation-system

Jul 27, 2026

Commits on Jul 27, 2026

Add the lecture evaluation system: rubric engine, calibration, worked cases

xuanguang-li
authored and
mmcky
committed
Integrate the evaluation system into the plugin layout

mmcky
and
claude
committed
scoring: verdict from the rounded total; fix stale paths in docstring/template

mmcky
and
claude
committed
Persist the headline metrics; fix two static-metrics inconsistencies

mmcky
and
claude
committed
Document the reference examples: logic check and provenance

mmcky
and
claude
committed
Final-review fixes: harden run_all, share the env stamp, doc consistency

mmcky
and
claude
committed
Docs: skill usage, repo setup, and validated triage mode (#6)

mmcky
and
claude
committed
Rubric v2: enforced couplings, no-conversion verdict, sensitivity stamp, K-repeat

mmcky
and
claude
committed
Skill wiring: plugin-root anchoring and workspace evaluation directory

mmcky
and
claude
committed
Docs: local testing tiers and the switch back to the production marketplace

mmcky
and
claude
committed
Validation run: ge_arrow re-evaluated from a fresh checkout, verdict reproduced

mmcky
and
claude
committed
Docs: hands-on evaluation tutorial built on the ge_arrow validation run

mmcky
and
claude
committed
Docs: AGENTS.md as the canonical agent guide, led by single source of truth

mmcky
and
claude
committed
Install fix: co-located plugin sources, required owner, validator guards

mmcky
and
claude
committed
Validator: report manifest errors instead of crashing on them

mmcky
and
claude
committed
Line endings: normalize the two CRLF files, add .gitattributes

mmcky
and
claude
committed
Scoring: validate evidence before scoring; honest sensitivity stamp

mmcky
and
claude
committed
Rubric: matches_under_x64 caps correctness on its own (review C3)

mmcky
and
claude
committed
Fixtures: synthetic evidence pinning the rubric v2 paths (review B1)

mmcky
and
claude
committed
CI: scorecards must reproduce from evidence (review B2)

mmcky
and
claude
committed
Docs: single-source the baselines; state the new rubric behaviour

mmcky
and
claude
committed
Reviews: the PR #5 review of record, behind the filed item numbers

mmcky
and
claude
committed
Docs: loosen the skill-naming rule rather than replace it
mmcky
committed
Docs: catalog what shipped, and hold the conventions loosely
mmcky
committed
Docs: fold the audit plugin into the rebased docs
mmcky
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Land the lecture evaluation system (benchmark plugin 0.3.0: rubric v2, skill wired) - #5

Land the lecture evaluation system (benchmark plugin 0.3.0: rubric v2, skill wired)#5
mmcky merged 25 commits into
mainfrom
land-evaluation-system

Commits on Jul 27, 2026

Add the lecture evaluation system: rubric engine, calibration, worked cases

Integrate the evaluation system into the plugin layout

scoring: verdict from the rounded total; fix stale paths in docstring/template

Persist the headline metrics; fix two static-metrics inconsistencies

Document the reference examples: logic check and provenance

Final-review fixes: harden run_all, share the env stamp, doc consistency

Docs: skill usage, repo setup, and validated triage mode (#6)

Rubric v2: enforced couplings, no-conversion verdict, sensitivity stamp, K-repeat

Skill wiring: plugin-root anchoring and workspace evaluation directory

Docs: local testing tiers and the switch back to the production marketplace

Validation run: ge_arrow re-evaluated from a fresh checkout, verdict reproduced

Docs: hands-on evaluation tutorial built on the ge_arrow validation run

Docs: AGENTS.md as the canonical agent guide, led by single source of truth

Install fix: co-located plugin sources, required owner, validator guards

Validator: report manifest errors instead of crashing on them

Line endings: normalize the two CRLF files, add .gitattributes

Scoring: validate evidence before scoring; honest sensitivity stamp

Rubric: matches_under_x64 caps correctness on its own (review C3)

Fixtures: synthetic evidence pinning the rubric v2 paths (review B1)

CI: scorecards must reproduce from evidence (review B2)

Docs: single-source the baselines; state the new rubric behaviour

Reviews: the PR #5 review of record, behind the filed item numbers

Docs: loosen the skill-naming rule rather than replace it

Docs: catalog what shipped, and hold the conventions loosely

Docs: fold the audit plugin into the rebased docs