Add generic RPG runtime harness

backlog/tasks/task-12017 - Design-RPG-campaign-session-runtime-harness.md

@@ -0,0 +1,63 @@
+---
+id: TASK-12017
+title: Design RPG campaign/session runtime harness
+status: Done
+created_date: 2026-06-25 02:10
+labels:
+- design
+- rpg
+- ttrpg
+- backend
+documentation:
+- Docs/superpowers/specs/2026-06-25-rpg-campaign-session-runtime-design.md
+modified_files:
+- Docs/superpowers/specs/2026-06-25-rpg-campaign-session-runtime-design.md
+- backlog/tasks/task-12017 - Design-RPG-campaign-session-runtime-harness.md
+updated_date: 2026-06-25 15:07
+---
+
+## Description
+
+<!-- SECTION:DESCRIPTION:BEGIN -->
+Brainstorm and document a backend-first RPG/TTRPG harness design for tldw_server. Scope: core campaign/session runtime, append-only event ledger, cached snapshots, rules adapters for D&D 5e SRD/PF2e/Fate, hybrid rules-pack retrieval, REST and MCP surfaces, authority policy, testing, and rollout. No implementation code in this task.
+<!-- SECTION:DESCRIPTION:END -->
+
+## Acceptance Criteria
+<!-- AC:BEGIN -->
+- [x] #1 Approved design is written to Docs/superpowers/specs/2026-06-25-rpg-campaign-session-runtime-design.md
+- [x] #2 Spec includes architecture, data model, REST/MCP API, service behavior, error handling, testing, rollout, non-goals, and licensing constraints
+- [x] #3 Spec self-review finds no placeholders, contradictions, or ambiguous core requirements
+<!-- AC:END -->
+
+## Implementation Plan
+
+<!-- SECTION:PLAN:BEGIN -->
+1. Capture approved brainstorming decisions in a design spec.
+2. Self-review for placeholders, contradictions, scope creep, and ambiguity.
+3. Run lightweight documentation verification.
+4. Commit the Backlog task update and design document.
+<!-- SECTION:PLAN:END -->
+
+## Implementation Notes
+
+<!-- SECTION:IMPLEMENTATION_NOTES:BEGIN -->
+Spec written to Docs/superpowers/specs/2026-06-25-rpg-campaign-session-runtime-design.md. Self-review performed for unresolved markers/placeholders, contradictions, ambiguous scope, and whitespace issues. Verification: `rg -n "TBD|TODO|FIXME|placeholder|unclear|\?\?\?|\[ \]" Docs/superpowers/specs/2026-06-25-rpg-campaign-session-runtime-design.md` returned no matches; `git diff --check -- Docs/superpowers/specs/2026-06-25-rpg-campaign-session-runtime-design.md "backlog/tasks/task-12017 - Design-RPG-campaign-session-runtime-harness.md"` passed. Bandit skipped because this task only adds documentation and Backlog metadata, with no Python code changes.
+Follow-up design review amendments added: pinned `dnd5e_srd` to SRD 5.1, deferred SRD 5.2.1/5.5e to a separate key, tightened PF2e bundled prose requirements, clarified the `RPG_DB.py` repository boundary, required atomic multi-event proposal apply, tightened committed roll randomness provenance, and added AuthNZ privilege registry rollout/testing coverage. Verification repeated: unresolved-marker scan returned no matches and `git diff --check` passed for the spec/task scope.
+Status corrected to Done after confirming acceptance criteria, Definition of Done, design spec, final summary, and verification notes were already complete. Follow-on implementation tasks TASK-12018 through TASK-12028 completed the approved runtime plan.
+<!-- SECTION:IMPLEMENTATION_NOTES:END -->
+
+## Final Summary
+
+<!-- SECTION:FINAL_SUMMARY:BEGIN -->
+Documented and amended the approved RPG/TTRPG campaign/session runtime design. The spec defines the backend-first architecture, append-only event ledger, cached snapshots, pinned D&D SRD 5.1/PF2e/Fate rules adapter boundary, conservative rules-content licensing constraints, REST and MCP surfaces, authority policy, service behavior, error handling, testing strategy, permission rollout, and phased implementation path. No implementation code was changed.
+<!-- SECTION:FINAL_SUMMARY:END -->
+
+## Definition of Done
+<!-- DOD:BEGIN -->
+- [x] #1 Acceptance criteria completed
+- [x] #2 Tests or verification recorded
+- [x] #3 Documentation updated when relevant
+- [x] #4 Bandit run for touched code when applicable or document non-code/environment skip
+- [x] #5 Final summary added
+- [x] #6 Known skips or blockers documented
+<!-- DOD:END -->

backlog/tasks/task-12026 - Add-RPG-rules-lookup-and-session-context-builder.md

@@ -0,0 +1,79 @@
+---
+id: TASK-12026
+title: Add RPG rules lookup and session context builder
+status: Done
+created_date: 2026-06-25 04:14
+labels:
+- rpg
+- ttrpg
+- backend
+- api
+- implementation
+priority: high
+references:
+- TASK-12018
+- TASK-12024
+- TASK-12025
+documentation:
+- Docs/superpowers/plans/2026-06-25-rpg-campaign-session-runtime-implementation-plan.md
+modified_files:
+- Docs/superpowers/plans/2026-06-25-rpg-campaign-session-runtime-implementation-plan.md
+- tldw_Server_API/app/core/RPG/models.py
+- tldw_Server_API/app/core/RPG/rules/content_packs.py
+- tldw_Server_API/app/core/RPG/rules/lookup.py
+- tldw_Server_API/app/core/RPG/context.py
+- tldw_Server_API/app/core/RPG/service.py
+- tldw_Server_API/app/api/v1/schemas/rpg_schemas.py
+- tldw_Server_API/app/api/v1/endpoints/rpg.py
+- tldw_Server_API/tests/RPG/test_rpg_rules_context.py
+- tldw_Server_API/tests/RPG/test_rpg_api.py
+- tldw_Server_API/tests/fixtures/privilege_route_registry_snapshot.json
+updated_date: 2026-06-25 04:22
+---
+
+## Description
+
+<!-- SECTION:DESCRIPTION:BEGIN -->
+Implement the rules lookup and bounded session context slice for the RPG runtime, including citation-only bundled lookup behavior, context text/diagnostics, service methods, REST schemas/endpoints for rules lookup and context building, tests, and plan updates.
+<!-- SECTION:DESCRIPTION:END -->
+
+## Acceptance Criteria
+<!-- AC:BEGIN -->
+- [x] #1 Rules lookup returns citation metadata without bundled PF2E prose
+- [x] #2 Session context builder includes session/snapshot/rules data and respects max character budget
+- [x] #3 RPGService exposes lookup_rules and build_context using owner-scoped session state
+- [x] #4 REST rules/context endpoints work with focused API tests
+- [x] #5 Focused rules/context/API tests, compileall, Bandit, and diff checks pass
+<!-- AC:END -->
+
+## Implementation Plan
+
+<!-- SECTION:PLAN:BEGIN -->
+1. Write failing core and API tests for citation-only rules lookup, bounded session context, service methods, and REST endpoints.
+2. Implement citation/result dataclasses and lookup service with citation-only bundled results.
+3. Implement a bounded, incremental context builder and wire owner-scoped service methods.
+4. Add typed REST schemas and session-scoped lookup/context endpoints; regenerate the privilege route registry snapshot.
+5. Run focused RPG/API/privilege tests, compileall, Bandit, and diff checks, then commit.
+<!-- SECTION:PLAN:END -->
+
+## Implementation Notes
+
+<!-- SECTION:IMPLEMENTATION_NOTES:BEGIN -->
+Implemented citation-only rules lookup and bounded context building. Post-review fixes included: context assembly now stops incrementally at budget instead of slicing large output, REST responses use typed citation/item/diagnostic schemas, REST `max_chars` requires 1000..24000, service callers are clamped into that range, lookup diagnostics mark `result_mode="citation_index"`, and the privilege route registry snapshot was regenerated to include the new rules/context routes. TDD RED confirmed before implementation: focused tests failed with `ModuleNotFoundError: No module named 'tldw_Server_API.app.core.RPG.context'`. Verification: focused rules/context + RPG API tests passed (11 passed); full focused RPG plus privilege catalog/snapshot suite passed (48 passed); compileall passed; Bandit reported 0 results; git diff --check passed.
+<!-- SECTION:IMPLEMENTATION_NOTES:END -->
+
+## Final Summary
+
+<!-- SECTION:FINAL_SUMMARY:BEGIN -->
+Added citation-only rules lookup, bounded session context generation, owner-scoped service methods, and session-scoped REST endpoints for RPG rules lookup/context. The API now returns typed citation/context diagnostics, and the AuthNZ route snapshot includes the new routes.
+<!-- SECTION:FINAL_SUMMARY:END -->
+
+## Definition of Done
+<!-- DOD:BEGIN -->
+- [x] #1 Acceptance criteria completed
+- [x] #2 Tests or verification recorded
+- [x] #3 Documentation updated when relevant
+- [x] #4 Bandit run for touched code when applicable or document non-code/environment skip
+- [x] #5 Final summary added
+- [x] #6 Known skips or blockers documented
+<!-- DOD:END -->

backlog/tasks/task-12028 - Document-RPG-runtime-and-run-final-verification.md

@@ -0,0 +1,64 @@
+---
+id: TASK-12028
+title: Document RPG runtime and run final verification
+status: Done
+created_date: 2026-06-25 04:47
+labels:
+- rpg
+- ttrpg
+- backend
+- docs
+- verification
+priority: high
+references:
+- TASK-12027
+documentation:
+- Docs/superpowers/plans/2026-06-25-rpg-campaign-session-runtime-implementation-plan.md
+modified_files:
+- tldw_Server_API/app/core/RPG/README.md
+- Docs/superpowers/plans/2026-06-25-rpg-campaign-session-runtime-implementation-plan.md
+updated_date: 2026-06-25 04:53
+---
+
+## Description
+
+<!-- SECTION:DESCRIPTION:BEGIN -->
+Complete Task 10 for the RPG runtime plan: add the RPG runtime README, run focused and adjacent regression checks, run Bandit/compile sanity checks, record outcomes, and commit the documentation closeout.
+<!-- SECTION:DESCRIPTION:END -->
+
+## Acceptance Criteria
+<!-- AC:BEGIN -->
+- [x] #1 RPG runtime README documents scope, storage model, adapters, rules-pack/legal boundary, authority/proposal model, REST/MCP surfaces, and explicitly says it is not a VTT canvas/map/token system
+- [x] #2 Focused RPG suite passes
+- [x] #3 Adjacent VN Play, MCP idempotency/category, and privilege catalog tests pass or any blocker is documented with exact reason
+- [x] #4 Bandit and compileall checks pass for touched RPG Python scope
+- [x] #5 Backlog task and implementation plan record final verification outcomes
+<!-- AC:END -->
+
+## Implementation Plan
+
+<!-- SECTION:PLAN:BEGIN -->
+['Review current RPG implementation and plan Task 10 for documentation risks', 'Write the RPG runtime README using existing module behavior, not speculative features', 'Run focused and adjacent regression/security checks', 'Update Backlog and plan with results, then commit']
+<!-- SECTION:PLAN:END -->
+
+## Implementation Notes
+
+<!-- SECTION:IMPLEMENTATION_NOTES:BEGIN -->
+Added RPG runtime README documenting backend harness scope, non-VTT boundary, per-user ChaCha storage, event/snapshot/idempotency behavior, adapters and citation-first rules lookup, authority/proposal flow, REST/MCP surfaces, concrete input limits, and current non-goals. Subagent documentation review identified and was used to correct over-broad optimistic-sequence wording, overclaiming around user rules-pack/RAG lookup, and missing concrete REST/MCP limits. Verification passed: python -m pytest tldw_Server_API/tests/RPG -q => 59 passed; python -m pytest tldw_Server_API/tests/VN_Play/test_vn_play_db.py tldw_Server_API/app/core/MCP_unified/tests/test_idempotency_and_category.py tldw_Server_API/tests/PrivilegeCatalog/test_endpoint_scope_catalog_sync.py -q => 47 passed; compileall touched RPG scope passed; Bandit /tmp/bandit_rpg_runtime.json reported 0 results/errors/skips; git diff --check passed. No virtual tabletop canvas/map/token/wall/lighting/live renderer features were added.
+<!-- SECTION:IMPLEMENTATION_NOTES:END -->
+
+## Final Summary
+
+<!-- SECTION:FINAL_SUMMARY:BEGIN -->
+Documented the RPG runtime as a generic backend TTRPG harness and completed the final regression/security closeout. The README describes implemented storage, event, adapter, rules lookup, authority, REST, and MCP behavior while explicitly calling out non-goals and unimplemented surfaces.
+<!-- SECTION:FINAL_SUMMARY:END -->
+
+## Definition of Done
+<!-- DOD:BEGIN -->
+- [x] #1 Acceptance criteria completed
+- [x] #2 Tests or verification recorded
+- [x] #3 Documentation updated when relevant
+- [x] #4 Bandit run for touched code when applicable or document non-code/environment skip
+- [x] #5 Final summary added
+- [x] #6 Known skips or blockers documented
+<!-- DOD:END -->

backlog/tasks/task-12029 - Design-RPG-rules-pack-attachment-and-retrieval-backed-lookup.md

@@ -0,0 +1,78 @@
+---
+id: TASK-12029
+title: Design RPG rules-pack attachment and retrieval-backed lookup
+status: Done
+created_date: 2026-06-25 15:08
+labels:
+- design
+- rpg
+- ttrpg
+- rag
+- backend
+priority: high
+references:
+- TASK-12017
+- TASK-12026
+- TASK-12028
+- TASK-12030
+- TASK-12031
+documentation:
+- tldw_Server_API/app/core/RPG/README.md
+- Docs/superpowers/specs/2026-06-25-rpg-campaign-session-runtime-design.md
+- Docs/superpowers/specs/2026-06-25-rpg-rules-pack-attachment-retrieval-design.md
+- Docs/superpowers/plans/2026-06-25-rpg-rules-pack-attachment-retrieval-implementation-plan.md
+updated_date: 2026-06-26 03:34
+modified_files:
+- Docs/superpowers/specs/2026-06-25-rpg-rules-pack-attachment-retrieval-design.md
+- Docs/superpowers/plans/2026-06-25-rpg-rules-pack-attachment-retrieval-implementation-plan.md
+- backlog/tasks/task-12029 - Design-RPG-rules-pack-attachment-and-retrieval-backed-lookup.md
+- backlog/tasks/task-12030 - Implement-RPG-rules-pack-attachment-and-retrieval-backed-lookup.md
+---
+
+## Description
+
+<!-- SECTION:DESCRIPTION:BEGIN -->
+Design the next RPG runtime feature: attaching user-provided rules-pack references to campaigns/sessions and using existing retrieval/RAG infrastructure to augment RPG rules lookup and context building without copying long-form rules prose into RPG tables. Scope is design only; implementation should follow in separate tasks after approval.
+<!-- SECTION:DESCRIPTION:END -->
+
+## Acceptance Criteria
+<!-- AC:BEGIN -->
+- [x] #1 Design defines how campaigns and sessions attach, list, update, and remove user rules-pack references without duplicating rules prose into RPG tables
+- [x] #2 Design specifies how RPG rules lookup blends built-in citation-only references with user-provided retrieval/RAG results, including citation/attribution fields and ranking/fallback behavior
+- [x] #3 Design specifies how the session context builder includes retrieved user rules snippets within existing context bounds and diagnostics
+- [x] #4 Design covers REST and MCP surface changes, AuthNZ privileges, idempotency/concurrency behavior for attachment writes, and failure modes
+- [x] #5 Design documents licensing/privacy constraints for user-provided rules content and makes clear that bundled adapters remain mechanics-metadata/citation-only
+- [x] #6 Design includes a test and verification plan for repository/service/API/MCP behavior, retrieval mocking, privilege catalog sync, Bandit, and focused regression coverage
+<!-- AC:END -->
+
+## Implementation Plan
+
+<!-- SECTION:PLAN:BEGIN -->
+['Inspect existing RPG runtime and RAG/retrieval APIs to identify the safest integration boundary', 'Propose design options for rules-pack reference storage and retrieval execution', 'Write an approved design spec under Docs/superpowers/specs/', 'Self-review the spec for scope creep, ambiguous retrieval semantics, and licensing overclaims', 'After approval, create a separate implementation plan/task sequence']
+<!-- SECTION:PLAN:END -->
+
+## Implementation Notes
+
+<!-- SECTION:IMPLEMENTATION_NOTES:BEGIN -->
+2026-06-25: Brainstorming complete. Approved design direction: hybrid direct media/media-collection references now with registry-compatible schema later; sessions copy campaign refs at creation; rules lookup is snippet/citation-first with opt-in generated answer mode; misses fall back only to bundled citation-only references; no broad RAG/web fallback.
+2026-06-25: Wrote approved design spec at Docs/superpowers/specs/2026-06-25-rpg-rules-pack-attachment-retrieval-design.md. Self-review checked for placeholders, contradictory scope, ambiguous retrieval behavior, and licensing overclaims. `git diff --check` passed. Bandit is not applicable because this step changed only Markdown/backlog task metadata, not Python code.
+2026-06-25: Post-review amendment tightened the spec around async service boundaries, media.read requirements for attached-source dereference/retrieval, live collection readiness semantics, server-owned ref timestamps, answer-mode quota/governance, and authorization regression tests.
+2026-06-25: Wrote implementation plan at Docs/superpowers/plans/2026-06-25-rpg-rules-pack-attachment-retrieval-implementation-plan.md after checking current RPG, RAG, Media, MCP, and Chat integration points. Self-review scan found and removed ellipsis placeholders; `git diff --check` passed for the plan. Created follow-on implementation task TASK-12030. Bandit is not applicable for this step because only Markdown/backlog metadata changed.
+2026-06-25: Implementation completed in TASK-12030. Follow-up TASK-12031 tracks host-level MCP answer-generation controls so MCP answer mode can pass a trusted preflight marker instead of remaining fail-closed.
+<!-- SECTION:IMPLEMENTATION_NOTES:END -->
+
+## Final Summary
+
+<!-- SECTION:FINAL_SUMMARY:BEGIN -->
+Completed the approved RPG rules-pack attachment/retrieval design package and implementation handoff. The design spec defines storage, retrieval, answer-mode, REST/MCP, AuthNZ, licensing, and verification behavior; the implementation plan breaks the runtime work into TDD-backed repository, service, REST, retrieval, answer, context, MCP, and verification tasks. Runtime code changes are tracked separately in TASK-12030.
+<!-- SECTION:FINAL_SUMMARY:END -->
+
+## Definition of Done
+<!-- DOD:BEGIN -->
+- [x] #1 Acceptance criteria completed
+- [x] #2 Tests or verification recorded
+- [x] #3 Documentation updated when relevant
+- [x] #4 Bandit run for touched code when applicable or document non-code/environment skip
+- [x] #5 Final summary added
+- [x] #6 Known skips or blockers documented
+<!-- DOD:END -->