r3-publish batch: retire r2 vocabulary & references (on-v3-release)

DEVELOPER_GUIDE.md

@@ -40,13 +40,13 @@ fork/clone → branch → edit → validate → push → PR
      - Author a new prompt using local instructions:
 
        ```
-       MUST FULLY EXECUTE `instructions/r2/core/workflows/coding-agents-prompting-flow.md` to author a new R3 Rosetta <skill/agent/workflow/rule/prompt family> `<name>`: <description of what it should be>
+       MUST FULLY EXECUTE `instructions/r3/core/workflows/coding-agents-prompting-flow.md` to author a new R3 Rosetta <skill/agent/workflow/rule/prompt family> `<name>`: <description of what it should be>
        ```
 
      - Author a new prompt via Rosetta MCP:
 
        ```
-       MUST ACQUIRE coding-agents-prompting-flow.md FROM KB AND FULLY EXECUTE IT to author a new R3 Rosetta <skill/agent/workflow/rule/prompt family> `<name>`: <description of what it should be>
+       MUST USE FLOW coding-agents-prompting-flow.md to author a new R3 Rosetta <skill/agent/workflow/rule/prompt family> `<name>`: <description of what it should be>
        ```
 
      Include in every prompt-change PR: a prompt brief (goal, non-goals, constraints), before/after behavior examples, and validation evidence (attach to the PR description).

OVERVIEW.md

@@ -19,7 +19,7 @@ Design principles:
 
 **Agent-agnostic.** Works across Cursor, Claude Code, VS Code, Windsurf, JetBrains (Copilot, Junie), GitHub Copilot, Codex, Antigravity, OpenCode, and any MCP-compatible IDE. Adopts agent-specific features where available, simulates them where not.
 
-**Progressive disclosure.** Instructions load in stages (bootstrap, classification, workflow-specific, sub-instructions) to [prevent context overflow](docs/ARCHITECTURE.md#context-overflow-prevention). The agent gets only what it needs for the current task.
+**Progressive disclosure.** Instructions load in stages (bootstrap, classification, workflow-specific, sub-instructions) to [prevent context overflow](docs/MCP-ARCHITECTURE.md#context-overflow-prevention). The agent gets only what it needs for the current task.
 
 **Classification-first.** Every request is auto-classified into a [workflow type](USAGE_GUIDE.md#workflows) before any work begins. Classification drives which instructions, skills, and rules load. Provided workflows are used as templates.
 
@@ -98,7 +98,7 @@ Instructions are organized in three layers that merge at runtime:
 - **Organization** — your company's conventions and policies
 - **Project** — local repo docs and configs
 
-Layers at the same resource path get [bundled together](docs/ARCHITECTURE.md#bundler). This is layered customization, not multi-tenancy. See [Architecture](docs/ARCHITECTURE.md) for component details and data flow.
+Layers at the same resource path get [bundled together](docs/MCP-ARCHITECTURE.md#bundler). This is layered customization, not multi-tenancy. See [Architecture](docs/ARCHITECTURE.md) for component details and data flow.
 
 ## What Rosetta Does Not Do
 

REVIEW.md

@@ -86,7 +86,7 @@ Duplication in instructions is a maintenance and correctness hazard.
 
 - **Search for existing coverage.** Before approving a new instruction, verify no other file already covers the same topic.
 - **One canonical source.** If content exists in two places, pick one home and make the other a reference.
-- **Bundler-aware.** Core and org files at the same VFS resource path get [bundled](docs/ARCHITECTURE.md#bundler). Verify that overlapping paths complement, not contradict.
+- **Bundler-aware.** Core and org files at the same VFS resource path get [bundled](docs/MCP-ARCHITECTURE.md#bundler). Verify that overlapping paths complement, not contradict.
 
 ### CI Pipeline Checks
 

docs/ARCHITECTURE.md

@@ -90,151 +90,11 @@ Instructions flow up: files are published by the CLI into RAGFlow, served by Ros
 
 ## Rosetta MCP
 
-The MCP server is the guiding layer between IDEs and the knowledge base. It exposes guardrails and common best practices, and provides a structured menu of available instructions; the coding agent selects what it needs, and Rosetta delivers only those — preventing context overload. Published on PyPI as `ims-mcp`. Built on [FastMCP v3](https://gofastmcp.com/) (latest stable) with [OAuthProxy](https://gofastmcp.com/servers/auth/oauth-proxy) for authentication and [RAGFlow](https://ragflow.io/) as the document engine backend. Speaks in VFS resource paths, adds context headers describing what information means and how to use it, and controls context size automatically.
-MCP changes are validated with `pytest`, `validate-types.sh`, and the end-to-end `verify_mcp.py` integration check.
+MCP is the **secondary** delivery mode — plugins are primary ([Rosettify](#rosettify) generates and installs them). MCP serves teams that want centrally managed, always-fresh instructions with nothing copied into the repository. Published on PyPI as `ims-mcp`.
 
-**Transport options:**
-- **Streamable HTTP with OAuth** (default). Stateful: the server holds session state and can issue callbacks to the IDE. Zero local dependencies. Cursor, Claude Code, and Codex connect directly. When scaling to multiple replicas, sticky sessions are required (see [DEPLOYMENT_GUIDE.md](../DEPLOYMENT_GUIDE.md)).
-- **STDIO** for air-gapped environments. Runs `uvx ims-mcp` locally with API key auth.
+Server internals live in **[MCP-ARCHITECTURE.md](MCP-ARCHITECTURE.md)** — read it when you touch any of: the `ims-mcp` server (FastMCP v3), transports (Streamable HTTP + OAuth 2.1, STDIO), authentication and OAuth modes (`oauth`/`oidc`/`github`, token lifecycle, policy-based authorization), Redis schema migrations, VFS resource paths and auto-tagging (tag-based retrieval), the MCP tools (`get_context_instructions`, `query_instructions`, `list_instructions`) and the `rosetta://{path}` resource, document bundling (core + organization overlays, `sort_order`, `INSTRUCTION_ROOT_FILTER`), XML/flat listings, context overflow prevention (query list threshold, context headers), `mcp-files-mode.md` alias bindings, or `ACQUIRE … FROM KB` generated shells. MCP changes are validated with `pytest`, `validate-types.sh`, and the end-to-end `verify_mcp.py` integration check.
 
-**Authentication:** HTTP uses OAuth 2.1 via FastMCP's proxy layer (supports any provider: Keycloak, GitHub, Google, Azure). STDIO uses `ROSETTA_API_KEY`. Policy-based authorization: `aia-*` read-only, `project-*` configurable. For the two-leg proxy architecture, scope separation, and token lifecycle details, see [AUTHENTICATION.md](AUTHENTICATION.md).
-
-Three OAuth modes controlled by `ROSETTA_OAUTH_MODE`:
-
-**`oauth` mode** (default) — generic OAuth 2.0 with token introspection:
-
-| Env var | Purpose |
-|---|---|
-| `ROSETTA_OAUTH_AUTHORIZATION_ENDPOINT` | Upstream IdP authorization URL |
-| `ROSETTA_OAUTH_TOKEN_ENDPOINT` | Upstream IdP token URL |
-| `ROSETTA_OAUTH_INTROSPECTION_ENDPOINT` | Upstream IdP introspection URL |
-| `ROSETTA_OAUTH_CLIENT_ID` | Pre-registered IdP client ID |
-| `ROSETTA_OAUTH_CLIENT_SECRET` | IdP client secret |
-| `ROSETTA_OAUTH_BASE_URL` | Public URL of Rosetta MCP |
-| `ROSETTA_JWT_SIGNING_KEY` | Secret for signing FastMCP JWTs |
-| `ROSETTA_OAUTH_REVOCATION_ENDPOINT` | *(optional)* Token revocation URL |
-| `ROSETTA_OAUTH_CALLBACK_PATH` | *(optional)* Callback path (default: `/auth/callback`) |
-| `ROSETTA_OAUTH_REQUIRED_SCOPES` | *(optional)* Scopes required on tokens |
-| `ROSETTA_OAUTH_VALID_SCOPES` | *(optional)* Scopes advertised in `.well-known` |
-| `ROSETTA_OAUTH_EXTRA_SCOPES` | *(optional)* Scopes forwarded to IdP authorize endpoint |
-
-Upstream IdP issues opaque tokens; Rosetta introspects them on each request via `IntrospectionTokenVerifier`. Cached 15 min.
-
-**`oidc` mode** — OIDC auto-discovery with local JWT verification:
-
-| Env var | Purpose |
-|---|---|
-| `ROSETTA_OAUTH_OIDC_CONFIG_URL` | IdP OIDC discovery URL (`.well-known/openid-configuration`) |
-| `ROSETTA_OAUTH_CLIENT_ID` | Pre-registered IdP client ID |
-| `ROSETTA_OAUTH_CLIENT_SECRET` | IdP client secret |
-| `ROSETTA_OAUTH_BASE_URL` | Public URL of Rosetta MCP |
-| `ROSETTA_JWT_SIGNING_KEY` | Secret for signing FastMCP JWTs |
-| `ROSETTA_OAUTH_CALLBACK_PATH` | *(optional)* Callback path (default: `/auth/callback`) |
-| `ROSETTA_OAUTH_REQUIRED_SCOPES` | *(optional)* Scopes required on tokens |
-| `ROSETTA_OAUTH_EXTRA_SCOPES` | *(optional)* Scopes forwarded to IdP authorize endpoint |
-
-Rosetta fetches IdP endpoints automatically from the discovery doc; tokens are JWTs verified locally via JWKS. No per-request introspection calls.
-
-**`github` mode** — GitHub OAuth via [GitHubProvider](https://gofastmcp.com/integrations/github):
-
-| Env var | Purpose |
-|---|---|
-| `ROSETTA_OAUTH_CLIENT_ID` | GitHub OAuth App Client ID |
-| `ROSETTA_OAUTH_CLIENT_SECRET` | GitHub OAuth App Client Secret |
-| `ROSETTA_OAUTH_BASE_URL` | Public URL of Rosetta MCP (HTTPS required in production) |
-| `ROSETTA_JWT_SIGNING_KEY` | Secret for signing FastMCP JWTs |
-| `ROSETTA_OAUTH_CALLBACK_PATH` | *(optional)* Callback path (default: `/auth/callback`) |
-| `ROSETTA_OAUTH_REQUIRED_SCOPES` | *(optional)* Required GitHub scopes (default: `user`) |
-
-GitHub endpoints are hardcoded. Tokens are validated via the GitHub API (`https://api.github.com/user`). User identity is extracted from GitHub profile (login, name, email).
-
-All three modes issue FastMCP JWTs to MCP clients and store upstream tokens in Redis (encrypted with `FERNET_KEY`). MCP clients never see IdP tokens; the IdP never sees FastMCP JWTs.
-
-### Redis Schema Migrations
-
-`ims_mcp/migrations.py` runs sequential schema migrations against Redis on every server startup via the FastMCP lifespan hook. Migrations are numbered methods (`_migrate_to_N`); only those ahead of the stored version run.
-
-**Key details:**
-- Version tracked in `rosetta:redis-schema-version` (plain integer)
-- Distributed lock (`rosetta:migration-lock`, 60 s TTL) prevents concurrent runs across pods on rolling deploys
-- Each migration runs exactly once; safe to deploy to multiple replicas simultaneously
-- All migration activity logged at `INFO` level under `ims_mcp.migrations`
-
-**Current migrations:**
-
-| Version | What it does |
-|---|---|
-| 1 | Baseline no-op — marks pre-migration deployments as version 1 |
-| 2 | Flushes `mcp-oauth-proxy-clients:*` keys so DCR/CIMD clients re-register with correct `required_scopes` |
-
-**Adding a migration:** add `_migrate_to_N`, bump `LATEST_REDIS_SCHEMA_VERSION = N`, deploy.
-
-### VFS and Tags
-
-Everything MCP works with is VFS (virtual file system) resource paths. The CLI strips instruction root prefixes during publishing, so `core/skills/planning/SKILL.md` becomes `skills/planning/SKILL.md`. Files at the same resource path get bundled together.
-
-**Tags are the primary access mechanism.** Typed load aliases (`USE SKILL`, `READ RULE`, `APPLY PHASE`, ...) query by tags, which provides the most direct and fastest access. The CLI's auto-tagging was designed specifically for this: every folder name, filename, and composite pair/triple becomes a tag, so agents can request exactly what they need. Keyword search (`query_instructions(query=...)`) remains an MCP-level fallback for discovery.
-
-### MCP Tools
-
-Three tools and one resource are currently exposed to agents. Five write-data tools are implemented but disabled (`@mcp.tool` registration commented out in `server.py`; the feature is retired but the code is kept in case it's needed again):
-
-| Tool | Purpose |
-|---|---|
-| `get_context_instructions` | MCP bootstrap gate: load `bootstrap-alwayson.md` |
-| `query_instructions` | Fetch instruction docs by tags (primary) or keyword search (fallback) |
-| `list_instructions` | Browse the VFS hierarchy (flat listing of immediate children) |
-| `query_project_context` *(disabled)* | Search project-specific docs in a target repo dataset |
-| `store_project_context` *(disabled)* | Create or update a document in a project dataset |
-| `discover_projects` *(disabled)* | List readable project datasets |
-| `plan_manager` *(disabled)* | Manage execution plans with phases, steps, dependencies, status. Has a `help` command for plan creators (subagents don't need it). Stores plan in REDIS. |
-| `submit_feedback` *(disabled)* | Auto-submit structured feedback on agent sessions |
-
-**Resource:** `rosetta://{path}` reads bundled instruction documents by VFS resource path.
-
-### Bundler
-
-The Bundler merges multiple documents at the same VFS resource path into a single XML response. When an agent loads a skill (`USE SKILL`), core and organization files at that path are concatenated into one payload:
-
-```xml
-<rosetta:file id="..." dataset="..." path="skills/planning/SKILL.md" name="..." tags="..." frontmatter="...">
-  [document content from core]
-</rosetta:file>
-<rosetta:file id="..." dataset="..." path="skills/planning/SKILL.md" name="..." tags="..." frontmatter="...">
-  [document content from organization overlay]
-</rosetta:file>
-```
-
-Documents sorted by `sort_order` (default: 1000000), then by name. `INSTRUCTION_ROOT_FILTER` controls which layers are included (e.g., `CORE,GRID`).
-
-### Listing
-
-Listing shows what exists in the VFS without loading content. Implemented by `list_instructions` to browse the instruction hierarchy. Two formats:
-
-**XML format** (default) includes metadata attributes:
-```xml
-<rosetta:folder dataset="..." path="skills/" />
-<rosetta:folder dataset="..." path="rules/" />
-<rosetta:file id="..." path="skills/planning/SKILL.md" name="..." tag="skills/planning/SKILL.md" frontmatter="..." />
-```
-
-**Flat format** returns resource paths only:
-```
-skills/planning/SKILL.md
-skills/coding/SKILL.md
-rules/guardrails.md
-```
-
-A full instruction suite listing is ~400 tokens. Frontmatter attributes (extracted by CLI during publishing) let agents understand document purpose from the listing alone, without follow-up reads.
-
-### Context Overflow Prevention
-
-MCP manages context size through two mechanisms:
-
-- **Query list threshold (5).** When `query_instructions` matches 5 or fewer documents, MCP returns full bundled content. When more than 5 match, it returns a listing instead, with a header guiding the agent to load specific files by their unique tags. This keeps responses bounded regardless of knowledge base size.
-- **Context headers.** Every MCP response includes a descriptive header explaining what the returned information is and how to act on it.
-
-### Command Aliases
+## Command Aliases
 
 Command aliases are used exclusively for Rosetta resources (instructions, knowledge base). Workspace files in the target repository (`docs/CONTEXT.md`, `agents/IMPLEMENTATION.md`, etc.) are read directly from the filesystem. This boundary is intentional: when an agent sees a typed alias (`USE SKILL ...`, `READ RULE ...`), it knows it is loading Rosetta instructions through the active mode; when it reads a file, it knows it is working with target repository files.
 
@@ -260,7 +120,7 @@ Instructions never call MCP tools directly. Rosetta defines command aliases that
 
 Verbs: `READ` = load into context; `APPLY` = load + fully execute; `USE`/`INVOKE` = activate. In plugin mode the typed aliases need NO mapping — they operate natively on the plugin files; the MCP mode file (`mcp-files-mode.md`: `query_instructions`/`list_instructions` by path-based tags) and local mode file (`local-files-mode.md`: reads from `instructions/r3`) map each alias to their mechanisms. In MCP, typed loads resolve via VFS resource paths (filename, parent/filename, or grandparent/parent/filename); LIST preferred when the folder is known.
 
-### Bootstrap Flow
+## Bootstrap Flow
 
 The runtime footprint is minimal: `bootstrap-alwayson.md` (core policies, `reasonable`, tasks, skill engagement, core files) plus exactly one mode file. MCP and local mode files bind command aliases to their mechanisms; plugin mode needs no mapping because aliases operate natively on plugin files. Everything heavy loads on demand behind skills and workflows:
 
@@ -287,10 +147,6 @@ The runtime footprint is minimal: `bootstrap-alwayson.md` (core policies, `reaso
 
 Requests are classified only when the user invokes `/rosetta`; a plain request legitimately runs lean. In MCP mode the agent calls `get_context_instructions` exactly once per session.
 
-**Key environment variables:** `ROSETTA_SERVER_URL`, `ROSETTA_API_KEY`, `INSTRUCTION_ROOT_FILTER`, `REDIS_URL`
-
-For MCP setup across all IDEs, see [Get Started](https://griddynamics.github.io/rosetta/#quick-start).
-
 ---
 
 ## RAGFlow (Rosetta Server)
@@ -481,7 +337,7 @@ Instructions Repo ──► CLI (publish) ──► RAGFlow ──► Rosetta MC
 
 ### Prerequisites
 
-- Python 3.12 (virtual environment at repo root: `venv/`)
+- Python 3.12 — ONE virtual environment at repo root: `venv/`. MUST be used for ALL Python code in this repo: every `src/*` package, tests, validation scripts, tools, ad-hoc runs. MUST NOT create any other venv (no `.venv`, no per-package venvs).
 - Pre-commit hook exists
 
 ### MCP and Server
@@ -685,6 +541,7 @@ After adding or changing instructions, publish with the CLI to make them availab
 
 ## Related Docs
 
+- [MCP Architecture](MCP-ARCHITECTURE.md) — `ims-mcp` server internals: transports, OAuth modes, Redis migrations, VFS/tags, tools, bundler, listings, overflow prevention
 - [Authentication](AUTHENTICATION.md) — two-leg OAuth proxy, scope architecture, token lifecycle, WARNING: very large document
 - [Developer Guide](../DEVELOPER_GUIDE.md) — repo navigation, where to change what
 - [Contributing](../CONTRIBUTING.md) — fastest path to a merged PR