Add generic RPG runtime harness

[rmusser01]

Summary

• Adds a generic backend RPG campaign/session harness for TTRPG orchestration, with event sourcing, deterministic snapshots, proposals, dice/check helpers, and adapters for D&D 5e SRD, Pathfinder 2e, and Fate.
• Adds rules-pack attachment and retrieval-backed lookup using user-owned media/media collections, with REST endpoints, privilege catalog entries, and scoped no-broad-fallback retrieval.
• Adds an optional RPG MCP module with read/write tools, explicit domain-permission checks, and fail-closed answer-mode governance pending TASK-12031.

Verification

• python -m pytest tldw_Server_API/tests/RPG tldw_Server_API/tests/Privileges/test_privilege_introspection.py::test_privilege_registry_snapshot_matches_live_app tldw_Server_API/tests/PrivilegeCatalog/test_endpoint_scope_catalog_sync.py -q -> 176 passed, 425 warnings.
• python -m ruff check on the new RPG code/test scope -> All checks passed.
• Full touched-file Ruff still reports pre-existing lint debt in shared bootstrap/protocol files; not introduced by the RPG code path.
• python -m bandit -r <touched app scope> -f json -o /tmp/bandit_task12030_focused.json -> 0 findings, 0 errors.
• git diff --check -> clean.

Notes

• This replaces the overly broad draft branch/PR with a focused branch based on origin/main.
• Draft status is intentional: AI-generated PRs require a human-written Change summary before merge.

---

Summary by cubic

Adds a generic RPG runtime with event-sourced campaigns/sessions, deterministic snapshots, rules adapters, and retrieval-backed rules lookup using user-owned media. Exposes REST rpg endpoints and an optional MCP rpg module; answer generation over MCP remains disabled pending TASK-12031.

• New Features

  • Runtime: campaigns/sessions, append-only events, cached snapshots, proposals; deterministic request hashing.
  • Rules: adapters for D&D 5e SRD, PF2e, Fate; dice/check helpers.
  • Rules packs: attach media items or collections; scoped retrieval with citations and optional grounded answers (REST-only, gated).
  • API: rpg endpoints with RBAC, token-scope guards, and rate limits; JSON schemas for create/read/lookup/context.
  • MCP: optional rpg module (MCP_ENABLE_RPG_MODULE); answer mode fail-closed until TASK-12031.
  • Security: adds media.read and rpg scopes to the privilege catalog.
  • Docs/Tests: designs, plans, README, and a full test suite.

• Migration

  • Grant needed scopes: rpg.* and media.read.
  • Optionally set MCP_ENABLE_RPG_MODULE=true to load MCP tools.
  • No manual DB migrations; RPG tables initialize on first use per user.

^{Written for commit cce4045. Summary will update on new commits.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 650d5db1-7007-492d-863c-5a9b4d08b0ce

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The PR updates Docker build and startup behavior for single-user and multi-user container setups, adjusts GitHub Actions setup and workflow gates across CI and publishing, and adds or rewrites repository docs, ignore rules, and design assets.

Changes

Container build and startup

Layer / File(s)	Summary
Build context and compose files .dockerignore, Dockerfiles/Dockerfile.webui, Dockerfiles/Monitoring/docker-compose.monitoring.yml, Dockerfiles/docker-compose.*.yml, Dockerfiles/README.md	Docker ignore rules, compose definitions, and deployment docs change for single-user, multi-user, WebUI, and monitoring setups.
First-run bootstrap Dockerfiles/entrypoints/tldw-app-first-run.sh	The first-run script changes Python selection, env-file parsing, Postgres URL derivation, and multi-user admin bootstrap behavior.

GitHub Actions and release workflows

Layer / File(s)	Summary
Shared CI plumbing .github/actionlint.yaml, .github/actions/*, .github/workflows/actionlint.yml	Actionlint picks up the new config and an extra workflow target, while the shared setup actions add FFmpeg, pip-cache, and Postgres client gating changes.
VZ Linux host-gated smoke .github/workflows/vz-linux-host-gated.yml	A new self-hosted macOS ARM64 vz-linux smoke workflow verifies the host, runs the smoke script, summarizes evidence, and uploads artifacts.
Test gate updates .github/workflows/backend-required.yml, .github/workflows/coverage-required.yml, .github/workflows/e2e-required.yml, .github/workflows/e2e-smoke.yml, .github/workflows/jobs-suite.yml, .github/workflows/notes-remediation-targeted.yml, .github/workflows/onboarding-docs-gate.yml, .github/workflows/security-required.yml	Backend, coverage, smoke, and jobs workflows adjust FFmpeg settings, remove --maxfail=1, raise the coverage floor, and update a dependency-review pin.
Frontend gate updates .github/workflows/frontend-required.yml, .github/workflows/frontend-e2e-tiers.yml, .github/workflows/ui-playground-quality-gates.yml, .github/workflows/ui-watchlists-extension-e2e.yml	Frontend jobs add SKIP_WXT_PREPARE, pin Redis images, disable FFmpeg installation, and switch the extension E2E run to Chrome with minimal locales.
UX smoke and research workflows .github/workflows/frontend-ux-gates.yml, .github/workflows/ui-research-workspace-nightly.yml, .github/workflows/ui-research-workspace-parity.yml, .github/workflows/ui-workspace-playground-*.yml	The UX smoke gate adds mock OpenAI startup, /chat cockpit regression execution, log capture, and cleanup, and the research-workspace workflows are added while the older playground workflows are removed.
MCP Unified release flow .github/tests/test_mcp_unified_artifact_gate.py, .github/workflows/mcp-unified-rc.yml, .github/workflows/mcp-unified-publish.yml	The MCP Unified CI shim dynamically loads boundary tests, and the RC and publish workflows build and conditionally publish release candidates.
Release packaging and SBOM .github/workflows/publish-docker.yml, .github/workflows/publish-ghcr-main.yml, .github/workflows/publish-pypi.yml, .github/workflows/pypi-package.yml, .github/workflows/sbom.yml	Publishing workflows rename artifacts, tighten triggers, pin login actions, and update CycloneDX generation fallbacks.

Repository docs and guardrails

Layer / File(s)	Summary
Ignore rules and docs .gitignore, .impeccable/design.json, DESIGN.md, CHANGELOG.md	Repository ignore patterns change, the design-system schema/spec files are added or expanded, and the unreleased changelog entries are rewritten.
Process docs and hooks AGENTS.md, Codeslop-Vibecheck-SKILL.md, .pre-commit-config.yaml	Task-tracking guidance, the review framework, and the local pre-push hook are added or expanded.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

• rmusser01/tldw_server#2258 — Shares the composite action changes around setup-ffmpeg, setup-python-deps, and wait-for-postgres.
• rmusser01/tldw_server#2098 — Touches the same frontend-ux-gates.yml mock OpenAI and cockpit regression flow.
• rmusser01/tldw_server#2432 — Relates to the MCP Unified artifact gate and publish workflow surface.

Poem

I hopped through YAML, crisp and bright,
with carrots pinned just right at night.
The mock server hummed, the workflows sang,
through docker burrows, bells that rang.
Hoppy builds! 🐰

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description covers the feature and verification, but it does not follow the repo template or include the required checklist and risk/rollback sections.	Rewrite the PR description to match the template headings: Summary, Validation, UX Audit Checklist, Watchlists sections, and Risk & Rollback.
Docstring Coverage	⚠️ Warning	Docstring coverage is 11.11% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title matches the main change: a generic RPG runtime harness.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/rpg-runtime-focused

---

_{Comment @coderabbitai help to get the list of available commands.}

[gemini-code-assist]

Code Review

This pull request implements RPG rules-pack attachment and retrieval-backed lookup, allowing campaigns and sessions to link user-owned media items or collections as rules references. It introduces reference normalization, scoped retrieval, grounded answer generation, and parallel REST and MCP interfaces. The review feedback highlights three key areas for improvement: concurrently executing reference validations using asyncio.gather to reduce I/O latency, converting the _rules_service dependency into a yield-dependency to prevent resource leaks from unclosed retrievers, and defensively handling potential ValueError exceptions in rules_pack_ref_from_dict to avoid raw 500 errors on malformed reference IDs.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[qodo-code-review]

PR Summary by Qodo

Add event-sourced RPG runtime harness with rules-pack retrieval + optional MCP tools
✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

Description

• Add event-sourced RPG campaigns/sessions with deterministic snapshots and proposal-based review.
• Add rules-pack attachment + retrieval-scoped rules lookup with optional grounded answer mode.
• Expose secured REST + optional MCP tools with explicit RBAC/scope guards and new privileges.

Diagram

graph TD
  A["REST: /api/v1/rpg"] --> B["RPGService"] --> C["RPGRepository (SQLite)"]
  B --> D["RulesLookupService"] --> E["RulesRetrievalAdapter"] --> F["RAG Retriever"]
  D --> G["ChatRulesAnswerGenerator"]
  H["MCP RPGModule"] --> B

  subgraph Legend
    direction LR
    _api["API Router"] ~~~ _svc["Service"] ~~~ _db[("Database")] ~~~ _ext["External/Subsystem"]
  end

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Lean on existing RAG permissions only (drop explicit media.read)

• ➕ Fewer new privileges to manage
• ➕ Less policy surface area in the short term
• ➖ Harder to reason about minimum needed access for rules lookup
• ➖ Risk of accidental broad retrieval access beyond user-owned rules sources

2. Persist snapshots only (no full event log)

• ➕ Simpler schema and less write amplification
• ➕ Less complexity around proposals and optimistic sequencing
• ➖ Loses auditability/replay/debuggability
• ➖ Harder to support deterministic replays and reviewed model/MCP suggestions

3. Split rules-pack refs into a dedicated table (normalized relational model)

• ➕ Queryable refs, easier partial updates/constraints
• ➕ Avoids JSON list replacement patterns
• ➖ More migrations/joins and CRUD surface
• ➖ Current whole-list replacement matches optimistic versioning and idempotency model

Recommendation: Current approach is sound for a generic orchestration harness: event sourcing + deterministic snapshots supports replay/audit and proposal workflows, while scoped retrieval avoids broad fallback. The main tradeoff is complexity; keep it by enforcing strict guards (already present) and relying on the large test suite to prevent regressions.

Files changed (53) +16413 / -28

Enhancement (24) +5240 / -1

rpg.pyAdd secured RPG REST endpoints +448/-0

Add secured RPG REST endpoints

• Adds '/api/v1/rpg' endpoints for adapter listing, campaign/session creation, event recording, rules lookup, rules-pack ref get/replace, context building, and proposal apply/reject. Enforces RBAC, token scopes, rate limits, and additional gating for answer-mode (chat budget + scope + rate limit).

tldw_Server_API/app/api/v1/endpoints/rpg.py

rpg_schemas.pyAdd Pydantic schemas for RPG endpoints +192/-0

Add Pydantic schemas for RPG endpoints

• Defines request/response models for campaigns, sessions, event recording, rules lookup/diagnostics, rules-pack refs, context build responses, and proposal review payloads with strict validation bounds.

tldw_Server_API/app/api/v1/schemas/rpg_schemas.py

RPG_DB.pyAdd SQLite repository for RPG persistence +1152/-0

Add SQLite repository for RPG persistence

• Implements 'RPGRepository' with schema creation and persistence operations for campaigns, sessions, append-only events, deterministic snapshots, proposals, and idempotency replay/conflict detection. Stores JSON payloads for settings/refs/events/snapshots and enforces owner scoping + optimistic concurrency checks.

tldw_Server_API/app/core/DB_Management/RPG_DB.py

rpg_module.pyAdd optional MCP RPG module with strict permission checks +943/-0

Add optional MCP RPG module with strict permission checks

• Implements MCP tools for RPG read/write operations, explicit required-permission metadata, fail-closed validation, and gated answer-mode behavior pending host-level controls. Integrates with per-user DB paths and retrieval for rules lookup when enabled.

tldw_Server_API/app/core/MCP_unified/modules/implementations/rpg_module.py

protocol.pyExtend MCP request context to accept db_paths +4/-1

Extend MCP request context to accept db_paths

• Adds an optional 'db_paths' parameter and normalizes it into 'RequestContext' for module/tool use (e.g., RPG per-user DB resolution).

tldw_Server_API/app/core/MCP_unified/protocol.py

evidence_models.pyIntroduce shared retrieval evidence dataclasses +26/-0

Introduce shared retrieval evidence dataclasses

• Adds 'RetrievedEvidence' and 'DerivedEvidence' data contracts used by RPG rules retrieval (and potentially other retrieval workflows).

tldw_Server_API/app/core/RAG/rag_service/evidence_models.py

authority.pyAdd authority decision policy for commits vs proposals +35/-0

Add authority decision policy for commits vs proposals

• Implements source-aware commit/proposal decisions based on authority settings (user/system always commit; model/mcp/import conditional).

tldw_Server_API/app/core/RPG/authority.py

checks.pyAdd adapter check-resolution wrapper +13/-0

Add adapter check-resolution wrapper

• Provides a small wrapper validating adapters return a 'CheckResult' when resolving checks.

tldw_Server_API/app/core/RPG/checks.py

context.pyAdd bounded session context builder +97/-0

Add bounded session context builder

• Implements 'SessionContextBuilder' that composes a bounded text context from session metadata, snapshot highlights, and rules evidence/citations with truncation diagnostics.

tldw_Server_API/app/core/RPG/context.py

dice.pyAdd deterministic-friendly dice roller with bounds +102/-0

Add deterministic-friendly dice roller with bounds

• Adds 'DiceRoller' supporting NdM(+/-K) expressions and Fate dice, with strict bounds and injectable values for tests.

tldw_Server_API/app/core/RPG/dice.py

errors.pyAdd RPG domain exception types +17/-0

Add RPG domain exception types

• Defines base and specialized exceptions for not-found, validation, and conflict scenarios used across the runtime and API mapping.

tldw_Server_API/app/core/RPG/errors.py

events.pyAdd event envelope validation and canonical hashing +84/-0

Add event envelope validation and canonical hashing

• Defines supported event types and stable-ID requirements, validates payload size/shape, and provides canonical JSON hashing for idempotency payload fingerprints.

tldw_Server_API/app/core/RPG/events.py

models.pyAdd core RPG dataclasses (campaign/session/event/snapshot) +152/-0

Add core RPG dataclasses (campaign/session/event/snapshot)

• Introduces typed dataclasses for adapter metadata, dice/check results, campaign/session records, events, and snapshot state/records.

tldw_Server_API/app/core/RPG/models.py

proposals.pyAdd proposal record model +26/-0

Add proposal record model

• Defines the persisted proposal record shape used for reviewable state changes.

tldw_Server_API/app/core/RPG/proposals.py

reducer.pyAdd snapshot reducer for supported event types +90/-0

Add snapshot reducer for supported event types

• Implements deterministic reduction of validated events into 'RPGSnapshotState' for scene/actors/npcs/quests/etc.

tldw_Server_API/app/core/RPG/reducer.py

__init__.pyInitialize rules subpackage +1/-0

Initialize rules subpackage

• Adds package marker for RPG rules components.

tldw_Server_API/app/core/RPG/rules/init.py

adapters.pyAdd rules adapter registry and D&D/PF2e/Fate adapters +317/-0

Add rules adapter registry and D&D/PF2e/Fate adapters

• Defines adapter protocol/registry plus bundled static adapters exposing actor/check schemas, bounded check resolution, and adapter info metadata.

tldw_Server_API/app/core/RPG/rules/adapters.py

answering.pyAdd grounded answer generator for rules evidence +158/-0

Add grounded answer generator for rules evidence

• Implements optional answer generation that prompts the chat backend to return strict JSON and filters citation IDs to those present in evidence.

tldw_Server_API/app/core/RPG/rules/answering.py

content_packs.pyAdd lookup result contracts and bundled citation-only packs +125/-0

Add lookup result contracts and bundled citation-only packs

• Defines lookup item/result dataclasses and bundled citation-only references for SRD/Fate/PF2e adapters (no prose).

tldw_Server_API/app/core/RPG/rules/content_packs.py

lookup.pyAdd rules lookup orchestrator with diagnostics +122/-0

Add rules lookup orchestrator with diagnostics

• Combines bundled citations with optional scoped retrieval; optionally generates grounded answers from retrieved snippets and records diagnostics/failure states.

tldw_Server_API/app/core/RPG/rules/lookup.py

refs.pyAdd rules-pack reference model and normalization +182/-0

Add rules-pack reference model and normalization

• Defines rules-pack refs, whole-list replacement result model, and normalization/validation helpers (dedupe, metadata validation, timestamps).

tldw_Server_API/app/core/RPG/rules/refs.py

retrieval.pyAdd retrieval adapter that scopes RAG to attached media only +284/-0

Add retrieval adapter that scopes RAG to attached media only

• Validates enabled refs via a source validator, builds an allowed-media list, executes retrieval through the existing retriever API, and filters results to avoid broad fallback.

tldw_Server_API/app/core/RPG/rules/retrieval.py

source_validation.pyValidate rules-pack sources using media ownership and collection readiness +92/-0

Validate rules-pack sources using media ownership and collection readiness

• Implements 'RPGRulesSourceValidator' to ensure media items/collections are readable by the owner and only include ready collection items for retrieval.

tldw_Server_API/app/core/RPG/rules/source_validation.py

service.pyAdd RPGService orchestration layer +578/-0

Add RPGService orchestration layer

• Implements campaign/session creation with idempotency hashing, event recording with commit vs proposal routing, snapshot management, rules-pack ref replacement/listing, rules lookup, and context building with strict validation and replay support.

tldw_Server_API/app/core/RPG/service.py

Refactor (2) +42 / -0

auth_deps.pyAdd compatibility aliases for auth guards +7/-0

Add compatibility aliases for auth guards

• Introduces 'RequirePermission' and 'TokenScopeGuard' aliases to standardize guard naming across route modules.

tldw_Server_API/app/api/v1/API_Deps/auth_deps.py

__init__.pyExpose RPG public API with lazy imports +35/-0

Expose RPG public API with lazy imports

• Adds package exports and lazy attribute resolution for key runtime classes to reduce import coupling.

tldw_Server_API/app/core/RPG/init.py

Tests (12) +5487 / -27

test_rpg_api.pyAdd REST API tests for RPG endpoints and privilege catalog sync +749/-0

Add REST API tests for RPG endpoints and privilege catalog sync

• Covers endpoint permission dependencies, idempotency behavior, rules-pack refs endpoints, and ensures required scopes are present in the privilege catalog.

tldw_Server_API/tests/RPG/test_rpg_api.py

test_rpg_core_models_adapters.pyAdd core model/adapter unit tests +58/-0

Add core model/adapter unit tests

• Adds unit coverage for adapter registry behavior and core model expectations.

tldw_Server_API/tests/RPG/test_rpg_core_models_adapters.py

test_rpg_db.pyAdd RPGRepository persistence/idempotency/concurrency tests +426/-0

Add RPGRepository persistence/idempotency/concurrency tests

• Exercises schema initialization, owner scoping, idempotent session creation, rules-pack ref versioning conflicts, and repository behaviors around snapshots/events.

tldw_Server_API/tests/RPG/test_rpg_db.py

test_rpg_dice_checks.pyAdd dice roller and check resolution tests +129/-0

Add dice roller and check resolution tests

• Validates dice bounds/parsing and adapter check-resolution outputs with injected randomness for determinism.

tldw_Server_API/tests/RPG/test_rpg_dice_checks.py

test_rpg_events_reducer.pyAdd event validation and reducer tests +135/-0

Add event validation and reducer tests

• Ensures supported event types validate correctly and reduce deterministically into snapshot state.

tldw_Server_API/tests/RPG/test_rpg_events_reducer.py

test_rpg_mcp_module.pyAdd MCP RPG module tool and permission tests +960/-0

Add MCP RPG module tool and permission tests

• Covers tool registration, permission gating, DB-path requirements, rules-pack operations, lookup/context behaviors, and fail-closed answer-mode checks.

tldw_Server_API/tests/RPG/test_rpg_mcp_module.py

test_rpg_rules_answering.pyAdd rules answer-generation and citation filtering tests +266/-0

Add rules answer-generation and citation filtering tests

• Tests answer-mode status transitions, grounded prompt behavior, JSON parsing, and filtering to allowed snippet IDs.

tldw_Server_API/tests/RPG/test_rpg_rules_answering.py

test_rpg_rules_context.pyAdd context builder + lookup integration tests +359/-0

Add context builder + lookup integration tests

• Validates context size bounding/truncation diagnostics and that bundled citations are included without requiring user-provided prose.

tldw_Server_API/tests/RPG/test_rpg_rules_context.py

test_rpg_rules_refs.pyAdd rules-pack ref normalization tests +171/-0

Add rules-pack ref normalization tests

• Tests ref normalization, deduplication, metadata validation, and timestamp handling.

tldw_Server_API/tests/RPG/test_rpg_rules_refs.py

test_rpg_rules_retrieval.pyAdd scoped retrieval adapter tests +348/-0

Add scoped retrieval adapter tests

• Ensures disabled refs are skipped, collections resolve ready media IDs, retrieval is scoped to allowed IDs, and broad fallback is never used.

tldw_Server_API/tests/RPG/test_rpg_rules_retrieval.py

test_rpg_service.pyAdd RPGService behavior tests (proposals/snapshots/idempotency) +545/-0

Add RPGService behavior tests (proposals/snapshots/idempotency)

• Covers default model-to-proposal behavior, user commit behavior, proposal apply replay semantics, and rules-pack ref validation hooks.

tldw_Server_API/tests/RPG/test_rpg_service.py

privilege_route_registry_snapshot.jsonUpdate privilege-route registry snapshot for new endpoints +1341/-27

Update privilege-route registry snapshot for new endpoints

• Refreshes the snapshot used by privilege introspection tests to account for newly registered endpoints and dependencies.

tldw_Server_API/tests/fixtures/privilege_route_registry_snapshot.json

Documentation (11) +5480 / -0

2026-06-25-rpg-campaign-session-runtime-implementation-plan.mdImplementation plan for campaign/session runtime harness +2911/-0

Implementation plan for campaign/session runtime harness

• Adds a detailed step-by-step plan for implementing the event-sourced campaign/session runtime, persistence, API surfaces, and verification tasks.

Docs/superpowers/plans/2026-06-25-rpg-campaign-session-runtime-implementation-plan.md

2026-06-25-rpg-rules-pack-attachment-retrieval-implementation-plan.mdImplementation plan for rules-pack attachment + retrieval-backed lookup +972/-0

Implementation plan for rules-pack attachment + retrieval-backed lookup

• Documents the planned design and rollout steps for attaching rules packs, validating sources, retrieval scoping, and answer-mode governance.

Docs/superpowers/plans/2026-06-25-rpg-rules-pack-attachment-retrieval-implementation-plan.md

2026-06-25-rpg-campaign-session-runtime-design.mdDesign spec for RPG runtime harness +566/-0

Design spec for RPG runtime harness

• Introduces the design for campaigns/sessions, event envelopes, deterministic snapshots, proposals, and idempotency expectations.

Docs/superpowers/specs/2026-06-25-rpg-campaign-session-runtime-design.md

2026-06-25-rpg-rules-pack-attachment-retrieval-design.mdDesign spec for rules-pack refs + retrieval-scoped lookup +451/-0

Design spec for rules-pack refs + retrieval-scoped lookup

• Defines rules-pack reference model, ownership/validation expectations, retrieval scoping rules, and citation/answer response behavior.

Docs/superpowers/specs/2026-06-25-rpg-rules-pack-attachment-retrieval-design.md

task-12017 - Design-RPG-campaign-session-runtime-harness.mdBacklog task for runtime harness design +63/-0

Backlog task for runtime harness design

• Adds the task record for designing the RPG campaign/session runtime harness.

backlog/tasks/task-12017 - Design-RPG-campaign-session-runtime-harness.md

task-12026 - Add-RPG-rules-lookup-and-session-context-builder.mdBacklog task for rules lookup + context builder +79/-0

Backlog task for rules lookup + context builder

• Adds the task record tracking implementation of rules lookup and session context builder.

backlog/tasks/task-12026 - Add-RPG-rules-lookup-and-session-context-builder.md

task-12028 - Document-RPG-runtime-and-run-final-verification.mdBacklog task for documentation and verification +64/-0

Backlog task for documentation and verification

• Adds the task record for documenting the RPG runtime and running focused verification.

backlog/tasks/task-12028 - Document-RPG-runtime-and-run-final-verification.md

task-12029 - Design-RPG-rules-pack-attachment-and-retrieval-backed-lookup.mdBacklog task for rules-pack design +78/-0

Backlog task for rules-pack design

• Adds the task record for designing rules-pack attachment and retrieval-backed lookup.

backlog/tasks/task-12029 - Design-RPG-rules-pack-attachment-and-retrieval-backed-lookup.md

task-12030 - Implement-RPG-rules-pack-attachment-and-retrieval-backed-lookup.mdBacklog task for rules-pack implementation +108/-0

Backlog task for rules-pack implementation

• Adds the task record for implementing rules-pack attachment, validation, and retrieval-backed lookup.

backlog/tasks/task-12030 - Implement-RPG-rules-pack-attachment-and-retrieval-backed-lookup.md

task-12031 - Wire-host-level-MCP-answer-generation-controls-for-RPG-rules-lookup.mdBacklog task for MCP answer-generation controls +52/-0

Backlog task for MCP answer-generation controls

• Adds the task record for wiring host-level governance to allow/deny answer generation in MCP mode.

backlog/tasks/task-12031 - Wire-host-level-MCP-answer-generation-controls-for-RPG-rules-lookup.md

README.mdAdd RPG runtime documentation +136/-0

Add RPG runtime documentation

• Documents the storage model, events/snapshots, adapters, rules-pack refs, authority/proposals, REST endpoints, MCP tools, and explicit non-goals/boundaries.

tldw_Server_API/app/core/RPG/README.md

Other (4) +164 / -0

privilege_catalog.yamlAdd RPG + media.read privilege catalog entries +130/-0

Add RPG + media.read privilege catalog entries

• Adds new 'media.read' scope and an 'rpg' scope bucket plus RPG-specific read/manage/review scopes used by the new REST/MCP surfaces.

tldw_Server_API/Config_Files/privilege_catalog.yaml

server.pyRegister optional RPG MCP module via env flag +14/-0

Register optional RPG MCP module via env flag

• Adds conditional registration of the 'RPGModule' when 'MCP_ENABLE_RPG_MODULE' is enabled.

tldw_Server_API/app/core/MCP_unified/server.py

constants.pyAdd RPG constants and limits +15/-0

Add RPG constants and limits

• Defines adapter IDs, schema/reducer versions, supported source types, and key size limits for payload/context bounds.

tldw_Server_API/app/core/RPG/constants.py

main.pyRegister RPG router in main app +5/-0

Register RPG router in main app

• Imports and includes the new RPG router under the v1 API prefix and route-policy gated inclusion paths.

tldw_Server_API/app/main.py

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📜 Skill insights (0)

Context used

✅ Compliance rules (platform): 74 rules

1. RPGError not in exceptions.py ✓ Resolved 📘 Rule violation ⌂ Architecture

Description

New custom exception types are defined in tldw_Server_API/app/core/RPG/errors.py instead of being
centralized in tldw_Server_API/app/core/exceptions.py. This violates the rule requiring custom
exceptions to live in the centralized core exceptions module.

Code

tldw_Server_API/app/core/RPG/errors.py[R4-17]

+class RPGError(Exception):
+    """Base exception for RPG runtime errors."""
+
+
+class RPGNotFoundError(RPGError):
+    """Raised when an RPG resource cannot be found."""
+
+
+class RPGValidationError(RPGError):
+    """Raised when RPG input fails domain validation."""
+
+
+class RPGConflictError(RPGError):
+    """Raised when an RPG write conflicts with current state."""

Evidence

PR Compliance ID 224217 requires new custom exception classes inheriting from Exception to be
defined in /app/core/exceptions.py. The PR introduces RPGError, RPGNotFoundError,
RPGValidationError, and RPGConflictError in /app/core/RPG/errors.py instead.

Rule 224217: Centralize custom exceptions in core module
tldw_Server_API/app/core/RPG/errors.py[4-17]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Custom exceptions for the RPG feature are defined in `tldw_Server_API/app/core/RPG/errors.py`, but compliance requires new custom exceptions to be centralized in `tldw_Server_API/app/core/exceptions.py` and imported from there.

## Issue Context
This impacts consistency of exception handling across the codebase and makes it harder to implement uniform error mapping.

## Fix Focus Areas
- tldw_Server_API/app/core/RPG/errors.py[4-17]
- tldw_Server_API/app/core/exceptions.py[1-120]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. _read_dependencies missing return type ✓ Resolved 📘 Rule violation ✧ Quality

Description

The helper functions _read_dependencies and _write_dependencies are missing explicit return type
annotations. This violates the requirement that all new/modified functions include type hints for
parameters and return values.

Code

tldw_Server_API/app/api/v1/endpoints/rpg.py[R73-83]

+def _read_dependencies(*scopes: str):
+    primary_scope = scopes[0]
+    return [
+        Depends(rbac_rate_limit(primary_scope)),
+        Depends(RequirePermission(*scopes)),
+        Depends(TokenScopeGuard("rpg", require_if_present=True, endpoint_id=primary_scope, count_as="call")),
+    ]
+
+
+def _write_dependencies(*scopes: str):
+    return _read_dependencies(*scopes)

Evidence

PR Compliance ID 224215 requires explicit parameter and return type hints. In rpg.py,
_read_dependencies and _write_dependencies define parameters but omit -> ... return
annotations.

Rule 224215: Require type hints on all function parameters and return values
tldw_Server_API/app/api/v1/endpoints/rpg.py[73-83]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`_read_dependencies` and `_write_dependencies` lack explicit return type annotations.

## Issue Context
Compliance requires type hints on all function parameters and return values (including helper functions).

## Fix Focus Areas
- tldw_Server_API/app/api/v1/endpoints/rpg.py[73-83]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Context bypasses rules permission ✓ Resolved 🐞 Bug ⛨ Security

Description

POST /rpg/sessions/{session_id}/context only requires rpg.sessions.read + media.read, but
RPGService.build_context() performs a rules lookup when query is provided, allowing callers
without rpg.rules.read to retrieve rules evidence/citations. This is an authorization gap because
the endpoint can expose rules lookup results via the context payload without the intended rules-read
privilege.

Code

tldw_Server_API/app/api/v1/endpoints/rpg.py[R374-378]

+@router.post(
+    "/sessions/{session_id}/context",
+    response_model=RPGContextResponse,
+    dependencies=_read_dependencies(RPG_SESSIONS_READ, MEDIA_READ),
+)

Evidence

The endpoint’s declared dependencies omit rpg.rules.read, but the service implementation will call
lookup_rules() (rules lookup path) during context building when a query is provided. Since
rpg.rules.read is defined in the privilege catalog as the scope for viewing rules lookup results,
omitting it on the context endpoint creates an authZ bypass for that capability.

tldw_Server_API/app/api/v1/endpoints/rpg.py[374-392]
tldw_Server_API/app/core/RPG/service.py[420-449]
tldw_Server_API/Config_Files/privilege_catalog.yaml[119-135]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`POST /api/v1/rpg/sessions/{session_id}/context` can return rules lookup-derived content when `query` is provided, but the route dependencies do not require the `rpg.rules.read` permission.

## Issue Context
- The route currently enforces `rpg.sessions.read` and `media.read` only.
- `RPGService.build_context()` calls `lookup_rules(..., mode="lookup")` when `query` is non-empty, which can return retrieval-backed rule snippets/citations.
- The privilege catalog explicitly defines `rpg.rules.read`, indicating rules lookup results are meant to be privilege-gated.

## Fix Focus Areas
- tldw_Server_API/app/api/v1/endpoints/rpg.py[374-392]
- tldw_Server_API/tests/RPG/test_rpg_api.py[128-132]

## Suggested fix
1. Update the context route dependencies to include `RPG_RULES_READ` (e.g., `dependencies=_read_dependencies(RPG_SESSIONS_READ, RPG_RULES_READ, MEDIA_READ)`).
2. Update/extend the route-permission test to assert `rpg.rules.read` is required for the context endpoint.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

View more (2)

4. Tests missing pytest markers 📘 Rule violation ▣ Testability

Description

New pytest tests are added without any of the required markers (unit, integration,
external_api, or local_llm_service). This violates the requirement that all tests be explicitly
marked for test selection and CI policy enforcement.

Code

tldw_Server_API/tests/RPG/test_rpg_api.py[R81-96]

+def test_rpg_endpoint_scopes_are_cataloged():
+    catalog_path = _REPO_ROOT / "tldw_Server_API" / "Config_Files" / "privilege_catalog.yaml"
+    catalog_ids = load_catalog_scope_ids(catalog_path)
+    expected_ids = {
+        rpg_endpoint.CHAT_COMPLETIONS,
+        rpg_endpoint.RPG_RULES_READ,
+        rpg_endpoint.RPG_CAMPAIGNS_READ,
+        rpg_endpoint.RPG_CAMPAIGNS_MANAGE,
+        rpg_endpoint.RPG_SESSIONS_READ,
+        rpg_endpoint.RPG_SESSIONS_MANAGE,
+        rpg_endpoint.RPG_PROPOSALS_REVIEW,
+        rpg_endpoint.MEDIA_READ,
+    }
+
+    assert expected_ids <= catalog_ids  # nosec B101
+

Evidence

PR Compliance ID 380651 requires all tests to be decorated with exactly one accepted pytest marker.
The new test functions (e.g., test_rpg_endpoint_scopes_are_cataloged) have no @pytest.mark.*
decorator.

Rule 380651: Apply appropriate pytest markers to all tests
tldw_Server_API/tests/RPG/test_rpg_api.py[81-96]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
New tests in `tldw_Server_API/tests/RPG/test_rpg_api.py` are not decorated with any of the required pytest markers.

## Issue Context
Compliance requires each test to have exactly one accepted marker so suites can be selected and governed reliably.

## Fix Focus Areas
- tldw_Server_API/tests/RPG/test_rpg_api.py[81-110]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

5. rpg_schemas missing docstrings ✓ Resolved 📘 Rule violation ✧ Quality

Description

Newly added RPG schema and endpoint modules define Pydantic classes and functions without
module/class/function docstrings, violating the requirement that all new code includes non-empty
docstrings. This reduces maintainability and fails PR Compliance ID 224214 for docstring coverage on
new modules, classes, and functions.

Code

tldw_Server_API/app/api/v1/schemas/rpg_schemas.py[R1-14]

+from __future__ import annotations
+
+from datetime import datetime
+from typing import Any, Literal
+
+from pydantic import BaseModel, ConfigDict, Field, StrictBool
+
+
+class RPGCampaignCreateRequest(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    title: str = Field(min_length=1, max_length=200)
+    description: str | None = Field(default=None, max_length=4000)
+    default_adapter_key: str = Field(min_length=1, max_length=80)

Evidence

PR Compliance ID 224214 requires docstrings for new modules, classes, and functions. In
tldw_Server_API/app/api/v1/schemas/rpg_schemas.py, the file starts with imports and proceeds
directly to class RPGCampaignCreateRequest(BaseModel): without any preceding module docstring or
an in-class string literal docstring, demonstrating missing documentation at both module and class
levels. In tldw_Server_API/app/api/v1/endpoints/rpg.py, the module similarly begins with imports
(no module docstring), and _read_dependencies begins with executable statements rather than a
string-literal docstring, showing the function lacks the required docstring as well.

Rule 224214: Require docstrings for all modules, classes, and functions
tldw_Server_API/app/api/v1/schemas/rpg_schemas.py[1-14]
tldw_Server_API/app/api/v1/endpoints/rpg.py[1-4]
tldw_Server_API/app/api/v1/endpoints/rpg.py[73-79]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Newly added RPG-related modules are missing required non-empty docstrings at the module level and for newly defined Pydantic model classes and endpoint/helper functions, which violates PR Compliance ID 224214 and reduces maintainability/reviewability.

## Issue Context
Compliance requires docstrings for all new modules, classes, and functions/methods. The schema module starts with imports and defines `BaseModel` classes (e.g., `RPGCampaignCreateRequest`) without module/class docstrings, and the endpoint module starts with imports (no module docstring) and includes functions (e.g., `_read_dependencies`) that begin with executable statements instead of a docstring.

## Fix Focus Areas
- tldw_Server_API/app/api/v1/schemas/rpg_schemas.py[1-15]
- tldw_Server_API/app/api/v1/schemas/rpg_schemas.py[9-15]
- tldw_Server_API/app/api/v1/endpoints/rpg.py[1-4]
- tldw_Server_API/app/api/v1/endpoints/rpg.py[73-84]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[rmusser01]

Rebased this PR onto latest dev and updated the PR base to dev.

Review fixes included in afb2c9e:

• Rules-pack source validations now run concurrently with asyncio.gather.
• _rules_service is now a yield dependency and closes its MultiDatabaseRetriever.
• Malformed stored ref IDs such as media_item:abc now map to RPGValidationError("invalid_rules_pack_ref_source_id").
• While verifying the rebased branch, fixed the shared MCP runtime wrapper so module-level PermissionError keeps the authorization denial message instead of being sanitized to tool_execution_error.

Verification:

• pytest tldw_Server_API/tests/RPG ... privilege checks -q: 179 passed, 431 warnings
• touched-scope ruff check: passed
• touched-scope Bandit JSON /tmp/bandit_task12030_rebased.json: 0 findings, 0 errors
• git diff --check: clean

[rmusser01]

Updated PR #2530 after rebasing onto latest dev.

Commit: 49ab15f

Review follow-up fixes:

• Added RPG endpoint/schema docstrings.
• Added explicit return types for RPG route dependency helper builders.
• Centralized RPG exceptions in app/core/exceptions.py with compatibility exports from app/core/RPG/errors.py.
• Added module-level pytest markers across RPG tests.
• Tightened RPG context route permissions to require rpg.rules.read along with rpg.sessions.read and media.read; regenerated the privilege route snapshot.

Fresh verification on the rebased branch:

• python -m pytest tldw_Server_API/tests/RPG tldw_Server_API/tests/Privileges/test_privilege_introspection.py::test_privilege_registry_snapshot_matches_live_app tldw_Server_API/tests/PrivilegeCatalog/test_endpoint_scope_catalog_sync.py -q -> 179 passed, 431 warnings.
• python -m ruff check <RPG/router/MCP/exception touched scope> -> All checks passed.
• python -m bandit -r <RPG/router/MCP/exception touched app scope> -f json -o /tmp/bandit_task12030_rebased_qodo.json -> 0 findings, 0 errors.
• git diff --check -> clean.