[curriculum-eval] score-history.json: false "declining" trend for 2 files caused by checkpoint-emoji heading oscillation, not content regression #4052

File: workshop/side-quest-11-09-agent-session-phases.md and workshop/side-quest-12-01-iterate-agent-output.md (score-history pipeline)
Overall Score: both files flagged declining with delta: -2.00 in score-history.json (baseline 7.69/7.91 → latest 5.69/5.91), against a corpus mean of 6.09/10.0

Flagged Dimensions:

Root Cause (≤ 2 sentences):
Both files' -2.00 score drop is a false signal: git log shows their ## Checkpoint heading was changed from ✅ to :white_check_mark: in PR github/gh-aw-workshop#2553 (Aug 9), back to ✅ in PR #2734-adjacent work, then to :white_check_mark: again in PR #3677 (Sep 21) — each toggle flips CHECKPOINT_RE's match result and swings checkpoint_quality by its full weight (2.0), producing the appearance of declining instructional quality when no prose content changed at all. This is the same CHECKPOINT_RE literal-emoji bug reported separately, but it specifically corrupts the trend/regression-detection signal in score-history.json, which is more actionable (and more dangerous) than a one-time score error: it could cause an automated quality gate to flag a benign, guideline-compliant edit as a content regression requiring author attention.

Evidence (quoted from the file):

Commit 2406024 ("fix obvious guideline violations in 2 file(s)"): -## ✅ Checkpoint / +## :white_check_mark: Checkpoint for both side-quest-11-09-agent-session-phases.md and side-quest-12-01-iterate-agent-output.md. Commit 736ae04 ("replace ✅ shortcode with ✅ Unicode in all checkpoint headings") made the opposite change earlier in the same file's history.

Learning Science Rationale:
This is not a learning-science finding but a measurement-reliability finding: any trend-based curriculum-quality system must be robust to no-op formatting changes, or its regression alerts lose credibility and reviewers will learn to ignore them (alert fatigue) — undermining the entire purpose of automated trend tracking for catching real quality decay over time.

Improvement Prompt (for an agent):

After applying the CHECKPOINT_RE fix from the companion issue "curriculum_assessment.py: checkpoint_quality — regex misses :white_check_mark: gemoji heading used by 90/93 files" in .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py, re-run the scoring pipeline across the commit history window used by score-history.json and confirm that workshop/side-quest-11-09-agent-session-phases.md and workshop/side-quest-12-01-iterate-agent-output.md no longer show a declining trend — their score should be stable across the :white_check_mark:/✅ heading toggles in their git history once the regex recognizes both forms. Also check whether any other repository automation (e.g. a "fix guideline violations" bot workflow) is actively toggling checkpoint heading emoji style back and forth between commits, and if so, standardize on one form (:white_check_mark:, per this repo's style guide) to stop the oscillation at its source. Regenerate score-history.json after the fix.

Expected Score After Fix: Both files' trend direction changes from declining to stable, with latest_score rising from ~5.7–5.9 to an estimated ~7.7–7.9, matching their baseline_score (no real content change occurred).

Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · 137.7 AIC · ⊞ 10.4K · ◷

• expires on Oct 2, 2026, 1:08 PM UTC