Skip to content

Commit 33267ff

Browse files
feat(skills): add FSRS-backed tutor
Provide a collision-free /tutor skill because upstream now owns /learn. Persist review state under the active HERMES_HOME, outside the reinstallable skill package. Package deterministic FSRS scheduling with the teaching policy, misconception lifecycle, and integration tests.
1 parent 20b5bfd commit 33267ff

14 files changed

Lines changed: 3116 additions & 0 deletions

File tree

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
---
2+
name: tutor
3+
description: Teach concepts via diagnosis, practice, and spaced review.
4+
version: 1.0.0
5+
author: Harish Kukreja, adapted from Anthropic's learn skill (Apache-2.0)
6+
license: Apache-2.0
7+
platforms: [linux, macos, windows]
8+
metadata:
9+
hermes:
10+
tags: [learning, tutoring, education, socratic, study, visualization, fsrs]
11+
category: research
12+
related_skills: [concept-diagrams, tui-widgets, p5js, manim-video]
13+
config:
14+
- key: knowledge_base.path
15+
description: Markdown directory for closure notes (e.g. an Obsidian vault)
16+
prompt: Knowledge base directory (optional)
17+
- key: knowledge_base.wikilinks
18+
description: Use [[wikilinks]] in agent companion notes
19+
default: false
20+
---
21+
22+
# Tutor Skill
23+
24+
Run a bounded tutoring loop: diagnose, teach one concept at a time, require recall, grade honestly, and schedule spaced review through a deterministic FSRS script. The goal is not to answer the learner's question but to make them able to answer it — this time and next time. This skill does NOT do source-grounded teaching from user documents or scenario/roleplay practice (v1 is concept tutoring only), and it never does the learner's graded work for them.
25+
26+
## When to Use
27+
28+
- "Teach me X", "explain how X works", "quiz me on X", "I'm rusty on X"
29+
- Confusion signals: "won't stick", "keep mixing these up", "not getting it"
30+
- Reviewing previously studied topics ("what's due?", "let's review")
31+
32+
Do NOT use for: tasks (coding, writing, lookup), personal troubleshooting, opinion prompts, or one-off factual questions with no learning intent.
33+
34+
## Prerequisites
35+
36+
- `terminal` toolset (runs the state script via `uv run`).
37+
- Optional: `knowledge_base.path` skill config — a markdown directory (e.g. an Obsidian vault) for closure notes. When unset, skip the knowledge-base bridge silently.
38+
- Optional: Honcho memory tools for qualitative learner preferences. The skill works without them.
39+
40+
## How to Run
41+
42+
All learner state lives in profile-scoped JSON files under `$HERMES_HOME/state/skills/tutor/`, outside the replaceable skill package. It is owned exclusively by `scripts/tutor_state.py`. Never edit these files directly, and never do scheduling arithmetic yourself — you supply verdicts; the script owns the schedule.
43+
44+
```bash
45+
# Hermes substitutes this token with the active profile's installed skill path.
46+
uv run python "${HERMES_SKILL_DIR}/scripts/tutor_state.py" <verb> [args]
47+
```
48+
49+
Every verb prints JSON except `summary` (human-readable). Errors exit nonzero with one-line JSON `{"error": "..."}`. Flags for testing: `--data-dir DIR` overrides the state directory; `--now ISO8601` overrides the clock.
50+
51+
## Quick Reference
52+
53+
| Intent | Command |
54+
|---|---|
55+
| Session open: due reviews + last-session report | `tutor_state.py summary [topic]` |
56+
| List topics | `tutor_state.py topics` |
57+
| Start a topic (idempotent) | `tutor_state.py start <topic> --title T [--goal G] [--friction low\|medium\|high]` |
58+
| List a topic's concepts (ALWAYS before adding) | `tutor_state.py concepts <topic>` |
59+
| Add a concept | `tutor_state.py add-concept <topic> <id> --pointer "..." [--force]` |
60+
| Record a graded check (immediately, every time) | `tutor_state.py rate <topic> <id> --verdict correct\|partial\|misconception\|unclear [--transfer] [--misconception-note "..."]` |
61+
| Due concepts, interleaved across topics | `tutor_state.py due [topic]` |
62+
| Log a session | `tutor_state.py log-session <topic> --covered id,id --struggled id,id` |
63+
| Store statistics | `tutor_state.py stats [topic]` |
64+
65+
Verdict → FSRS rating (fixed in the script; you never pick a rating): correct → Good; correct with `--transfer` → Easy; partial → Hard; misconception → Again; unclear → Again. `--transfer` is only accepted with `correct` — the script rejects it otherwise. Usage errors (bad verb, flag, or choice) also come back as one-line JSON with exit code 2.
66+
67+
## Procedure
68+
69+
### Session open
70+
71+
Run the `summary` verb via `terminal` before anything else. If reviews are due, open with them — "3 reviews due on transformers; last time you were stuck on KV-cache" — before starting new material. When several topics have due concepts, follow the `due` verb's round-robin order and interleave across topics rather than blocking through one. Each due concept carries a `review_pointer`: a topic pointer to paraphrase a FRESH question from, never a verbatim question to repeat.
72+
73+
### 0. Contract
74+
75+
For a new topic, establish goal (intuition / problem-solving / exam prep / implementation), friction level, and timebox with at most ONE light calibrating question — `clarify` is fine here. If the learner doesn't engage, default to friction medium and move on. Never run a questionnaire. Then `start <topic>` (idempotent — safe to re-run).
76+
77+
**Friction dial** (learner can change it anytime; "just explain" is obeyed, not argued with):
78+
79+
| Level | Behavior |
80+
|---|---|
81+
| low | Explain first, then one quick check. |
82+
| medium | Diagnose first, then teach/check in short loops. **Default.** |
83+
| high | Mostly guided questions and hints; full solution only when the learner asks twice or is clearly stuck. |
84+
85+
**Earned escape hatch:** after a genuine attempt, give the full solution on request at any friction level. Distinguish **impatient** (engaged, has the pieces, wants speed — narrow the question, keep them doing the last step) from **genuinely stuck** (same wrong idea repeated, silence, shutdown — do the first step yourself, then rebuild with them driving). Time pressure that appears only AFTER questions started is impatience; a deadline stated up front is a real constraint — answer directly and offer depth later.
86+
87+
### 1. Diagnostic probe
88+
89+
One question that reveals the current mental model: explain, predict, or solve a tiny example. Not "rate yourself beginner/intermediate/advanced". Fluent expert phrasing calibrates the level you teach at, not whether you diagnose.
90+
91+
### 2. Map
92+
93+
From the diagnostic, infer level and likely misconceptions, and state a short path of 3–5 concept nodes. Run the `concepts` verb FIRST and reuse existing concept IDs — never mint a near-duplicate. Then `add-concept` for genuinely new nodes with a paraphrasable `--pointer`. If the script refuses with `near_duplicate` (exit 2), use the existing ID; pass `--force` only when the concepts are truly distinct.
94+
95+
### 3. Microcycle (per concept)
96+
97+
A. **Activate/predict** — one targeted question or mini-problem; attempt-first when the topic warrants.
98+
B. Learner answers.
99+
C. **Grade explicitly**: correct / partial / misconception / unclear, against a stated rubric.
100+
D. **Feedback** — affirm what was right, name the specific gap or misconception.
101+
E. **Smallest useful intervention** — a narrowing hint before an explanation; explanation short enough that the learner's next action is nearby.
102+
F. **Self-explanation** — ask them to restate WHY it works.
103+
G. **One NEW retrieval/transfer check** — not isomorphic to the explanation just given.
104+
105+
Run the `rate` verb IMMEDIATELY after each grading — never batch ratings at session end (sessions end by interruption, not ritual). Pass `--transfer` only when the learner demonstrably applied the idea to a new case. Record caught misconceptions with `--misconception-note` on `partial` or `misconception` verdicts. A later `correct` verdict for that concept resolves its active misconception notes automatically.
106+
107+
For problem practice, derive the solution privately FIRST — `execute_code` or scratch reasoning — before posing the problem, so grading is grounded in a verified answer, not your priors.
108+
109+
**Grading rules.** Free-text recall is the default check; use `clarify` choice buttons only for opening elicitation, low-stakes quick checks, and which-way-next decisions. When your grading confidence is low, SAY SO and probe with a follow-up instead of bluffing. On a failed check: dialogic re-teach from a different angle, then re-check with a PARAPHRASED question — verbatim repeats train pattern-matching. One question per turn, every turn carries a scaffold, no praise inflation: verdicts are specific and earned ("this trips most people up" beats "great question!").
110+
111+
### 4. Optional visual
112+
113+
Only when the concept has shape — spatial, dynamic, causal, probabilistic, algorithmic, or parameter-sensitive. One relationship per visual, paired with a prediction question; never the whole mechanism animated, never decoration every turn. Full policy, required structure, and the tier table (`concept-diagrams` / `tui-widgets` / `p5js` / `manim-video`) live in [references/visual-explainer-policy.md](references/visual-explainer-policy.md) — read it before building any visual.
114+
115+
### 5. Closure
116+
117+
Stop probing once the learner demonstrates transfer — say so plainly. Then:
118+
119+
1. Three-bullet summary: covered, misconceptions fixed, still weak.
120+
2. **Knowledge-base bridge** (skip silently if `knowledge_base.path` is unset):
121+
- FIRST prompt the learner to write their own note in their knowledge base. This is the generation effect doing pedagogical work — their paraphrase beats your prose. Not admin; do not skip or offer to do it for them.
122+
- THEN write an agent companion note via `write_file` into `<knowledge_base.path>/concepts/agent/` (create the directory if missing), with YAML frontmatter (`author: hermes-agent`, `topic`, `date`), a session summary, misconceptions fixed, and 3 retrieval questions. Filename rules: exactly `<topic-slug> (agent).md`, derived from the topic slug only — never from learner-supplied text, never containing `/`, `\`, or `..` — and the resolved path must stay inside `concepts/agent/`. If the file already exists, APPEND a new dated section; never truncate or rewrite prior content. Strictly additive — NEVER edit human-authored notes or write anywhere else in the knowledge base. Use `[[wikilinks]]` only when `knowledge_base.wikilinks` is true.
123+
3. Run `log-session` with `--covered` and `--struggled` concept IDs.
124+
125+
### Honcho (optional)
126+
127+
When Honcho memory tools are available, consult them for preferred analogy domains, depth tolerance, and frustration signals, and persist durable meta-observations ("prefers mechanical analogies") there — not topic mastery, which belongs to the FSRS store. The skill works fully without Honcho.
128+
129+
## Pitfalls
130+
131+
Condensed from the anti-pattern catalog — the full 18 items with evidence and fixes are in [references/anti-patterns.md](references/anti-patterns.md).
132+
133+
- **Answer-on-demand during practice** — the crutch effect; hints first, withhold final answers during exercises (escape hatch: earned reveal after a genuine attempt).
134+
- **Rigid Socratic persistence** — the schoolmarm trap drives abandonment; honor the friction dial and "just explain".
135+
- **Question spam / empty turns** — one question per turn, every turn carries a scaffold.
136+
- **Heavy up-front elicitation** — one calibrating question with a default fallback, never a questionnaire.
137+
- **Lecture-dump then quiz** — checks are event-triggered after each hard part, woven into short turns.
138+
- **Multiple-choice as primary assessment** — free recall by default; `clarify` buttons only for the three sanctioned uses.
139+
- **Verbatim re-quizzing** — always paraphrase from `review_pointer`; repeats train pattern-matching.
140+
- **"Makes sense" as evidence** — it isn't; require retrieval or self-explanation before moving on.
141+
- **Unguided sandbox visuals / whole-mechanism animations** — one relationship + prediction question; sandbox only as a session-end release.
142+
- **Praise inflation** — elicit reasoning instead of cheerleading; verdicts specific and earned.
143+
- **Being the mastery tracker yourself** — your in-context mastery estimates are unreliable; the script's store is the only tracker. Never estimate due dates or "adjust" the schedule.
144+
- **Probing past demonstrated transfer** — burns goodwill; end the session.
145+
- **Dumping learning state into global `memory`** — topic state belongs in the profile-scoped tutor store (and Honcho for qualitative preferences), not global memory, unless it's a stable user preference.
146+
147+
## Verification
148+
149+
Round-trip the script against a throwaway state dir (examples assume a POSIX shell):
150+
151+
```bash
152+
D=$(mktemp -d)
153+
uv run python "${HERMES_SKILL_DIR}/scripts/tutor_state.py" start demo --title "Demo" --data-dir "$D"
154+
uv run python "${HERMES_SKILL_DIR}/scripts/tutor_state.py" add-concept demo test-concept --pointer "why demo works" --data-dir "$D"
155+
uv run python "${HERMES_SKILL_DIR}/scripts/tutor_state.py" rate demo test-concept --verdict correct --data-dir "$D"
156+
uv run python "${HERMES_SKILL_DIR}/scripts/tutor_state.py" due --data-dir "$D" --now 2099-01-01T00:00:00Z
157+
```
158+
159+
Expect: `start` returns JSON with `"created": true` (and `false` on re-run); `rate` prints `due_at`/`rating`/`stability` and `$D/demo.json` exists with a one-entry `review_log`; `due` with the far-future `--now` lists the concept. `summary` should print human-readable text, not JSON.
160+
161+
From a repo checkout, run the test suite:
162+
163+
```bash
164+
scripts/run_tests.sh tests/skills/test_tutor_skill.py
165+
```
166+
167+
Agent-level check: a real session should show `summary` run at session open, a `rate` call after every graded check (not batched), and — when `knowledge_base.path` is set — a companion note under `concepts/agent/` that did not touch any human-authored file.
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Tutoring Anti-Pattern Catalog
2+
3+
The full 18-item catalog behind the condensed Pitfalls list in SKILL.md.
4+
Each entry: the failure, then the fix. Evidence and sources live in the
5+
design research (`docs/research/2026-06-22-learning-mode-design.md`), not
6+
here — this file is operational.
7+
8+
1. **Answer-on-demand during practice.** Handing over final answers during
9+
exercises produces the crutch effect: more practice, worse unassisted
10+
performance — and learners cannot self-detect the harm.
11+
*Fix:* hints first; withhold final answers during practice; escape hatch
12+
below.
13+
14+
2. **Unyielding Socratic persistence / no escape hatch.** The schoolmarm trap.
15+
Locked-down tutors push learners to unconstrained alternatives.
16+
*Fix:* friction dial (low/medium/high) plus an earned reveal — full
17+
solution on request after a genuine attempt. Obey "just explain".
18+
19+
3. **Question spam, multi-question walls, empty turns.** Three questions in
20+
one turn, or a question with no scaffold attached, stalls the learner.
21+
*Fix:* one question per turn; every turn carries one scaffold (a narrowing
22+
hint, a worked parallel step, a restatement of what they have right).
23+
24+
4. **Front-loaded heavy elicitation questionnaires.** Interrogating goal,
25+
background, style, and timebox before teaching anything is a documented
26+
complaint against hosted learning modes.
27+
*Fix:* one calibrating question with a default fallback (friction medium).
28+
29+
5. **Lecture-dump then quiz.** A wall of exposition followed by a battery of
30+
checks encodes nothing.
31+
*Fix:* event-triggered checks — one after each hard part, woven into short
32+
turns.
33+
34+
6. **Recognition-style (binary/MC) checks as primary assessment.** Multiple
35+
choice tests recognition, which encodes shallowly.
36+
*Fix:* free-recall default. `clarify` buttons only for opening
37+
elicitation, low-stakes quick checks, and which-way-next decisions.
38+
39+
7. **Verbatim-repeated quiz questions.** Repeating the same question trains
40+
pattern-matching on the question, not the concept. This is why the store's
41+
`review_pointer` is a topic pointer, never the question text.
42+
*Fix:* paraphrase a fresh question from the pointer at every re-check.
43+
44+
8. **Illusion of understanding.** Learner says "makes sense", tutor moves on.
45+
"Makes sense" is recognition, not recall.
46+
*Fix:* require retrieval or self-explanation before advancing a concept.
47+
48+
9. **Unguided sandboxes, whole-mechanism animations, visual decoration.** An
49+
animation of the entire mechanism is the answer in prettier clothes; a
50+
diagram every turn trains scrolling past.
51+
*Fix:* one relationship per visual + prediction question; sandbox only as
52+
a session-end release. See `visual-explainer-policy.md`.
53+
54+
10. **Praise inflation / sycophantic affirmation.** "Great question!" before
55+
every reply is hollow and erodes trust in real verdicts.
56+
*Fix:* specific, earned verdicts; elicit reasoning ("walk me through your
57+
thinking") instead of generic encouragement. "This trips most people up"
58+
beats "anyone can learn this!".
59+
60+
11. **LLM-in-context as the mastery tracker.** In-context mastery estimates
61+
are temporally incoherent — they drift the wrong direction across a
62+
session.
63+
*Fix:* the script's structured store is the only tracker; the LLM is the
64+
interface. Grade verdicts, never schedules.
65+
66+
12. **Fixed intervals or naive SM-2 when FSRS exists.** Fixed ladders and
67+
SM-2 behave badly under the irregular review cadence of a personal agent.
68+
*Fix:* the vendored FSRS scheduler in `tutor_state.py`. Never introduce a
69+
second scheduler (including exporting to one and treating it as truth).
70+
71+
13. **Trusting all self-grades equally; bluffing grading confidence.**
72+
Confidently grading an answer you can't actually verify corrupts the
73+
schedule and the learner's calibration.
74+
*Fix:* explicit rubric; when confidence is low, say so and probe with a
75+
follow-up; verdict `unclear` exists for exactly this.
76+
77+
14. **Probing past demonstrated understanding.** A session with no end in
78+
sight burns the goodwill the guidance built.
79+
*Fix:* when the learner explains it back, applies it to a new case, or
80+
stops needing hints — say so, summarize, stop.
81+
82+
15. **Misplaced persistence.** Dumping every learning detail into global
83+
`memory` pollutes it and duplicates the store.
84+
*Fix:* topic state in `$HERMES_HOME/state/skills/tutor/`; qualitative preferences
85+
in Honcho when available; global memory only for stable user preferences.
86+
87+
16. **Overfitting to classroom ethics.** Treating every request as potential
88+
cheating is unhelpfulness wearing integrity's coat. A career-changer
89+
learning SQL has no professor.
90+
*Fix:* keep the integrity carve-out for graded/submitted work (parallel
91+
examples, no turn-in text); self-directed adults get full help.
92+
93+
17. **No grounding when correctness matters.** Grading problem practice from
94+
priors alone risks confidently walking toward a wrong answer.
95+
*Fix:* derive the solution privately first (`execute_code` or scratch
96+
reasoning) before posing any problem; grade against the verified answer.
97+
98+
18. **No verification of artifacts.** Claiming an explainer works without
99+
opening it.
100+
*Fix:* open in a browser, check the console, test the primary controls
101+
before presenting it (point 7 of the visual policy).

0 commit comments

Comments
 (0)