Skip to content

Commit d3c612e

Browse files
authored
docs: remove leftover hiding-group and SingleSwitch references (#144)
#143 moved CLAUDE.md to MultiSwitch but left content describing the removed group-based KV-hiding mechanism, plus some wording that does not match the code: - Control Tokens: describe the token exchange (control token rewritten to its substitute id) instead of K=finfo.min group hiding. - Switch Configuration example: drop hiding_groups, hiding_policy, control_dims, lora_rank and lora_alpha, none of which GraniteSwitchConfig reads; show adapter_substitute_token_ids, max_lora_rank, adapter_ranks and the ms_* knobs instead. - Remove Gotcha 7 (hidden_count offset), which only applied to hiding groups. The remaining gotchas keep their numbers because tests cite them. - Project tree: drop the deleted hiding_constant_report.py; tests/audio/ holds only a .wav fixture, so describe it as such and remove it from the test-file placement list (audio tests live in unit/composer/integration/vllm). - <|base_reset|> exists only when composed with --base-reset-token. - HF backend: drop the Trainer/training claims; nothing is trainable. - `uv sync --group dev` is not "everything": note dev-latest and the tutorials extra. - Documentation index: add MULTISWITCH_EXPLAINED, AUDIO and CICD. Also fix two code comments that still referred to SingleSwitch (token_exchange.py, hf/modeling_granite_switch.py). Signed-off-by: aviv ron <rona@il.ibm.com>
1 parent 86bc033 commit d3c612e

3 files changed

Lines changed: 20 additions & 23 deletions

File tree

‎CLAUDE.md‎

Lines changed: 18 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,6 @@ granite-switch/
3838
│ │ ├── __init__.py
3939
│ │ ├── adapter_analysis.py
4040
│ │ ├── compose_report.py
41-
│ │ ├── hiding_constant_report.py
4241
│ │ ├── model_card.py
4342
│ │ └── population_table.py
4443
│ │
@@ -85,7 +84,7 @@ granite-switch/
8584
│ ├── vllm/ # vLLM-specific tests
8685
│ ├── composer/ # Compose system tests
8786
│ ├── integration/ # Cross-backend integration tests
88-
│ ├── audio/ # Audio/ASR tests (pytest -m audio)
87+
│ ├── audio/ # Audio test fixtures (sample .wav)
8988
│ └── shared/ # Shared test utilities and parametrized cases
9089
│
9190
├── .pre-commit/ # Pre-commit hook scripts (validate_links.py)
@@ -114,7 +113,9 @@ uv sync --extra vllm
114113
# With compose tools
115114
uv sync --extra compose
116115

117-
# Everything (development) -- `dev` is a dependency group, not an extra
116+
# Development (hf + compose + audio extras, pytest, vLLM 0.26) -- `dev` is a
117+
# dependency group, not an extra. `--group dev-latest` swaps in the newest vLLM line;
118+
# add `--extra tutorials` to run the tutorials.
118119
uv sync --group dev
119120
```
120121

@@ -157,7 +158,6 @@ from granite_switch.composer import GraniteSwitchComposer
157158
- **`tests/vllm/`**: vLLM implementation tests
158159
- **`tests/composer/`**: Compose system tests
159160
- **`tests/integration/`**: Cross-implementation and end-to-end integration tests
160-
- **`tests/audio/`**: Audio/ASR tests (`pytest -m audio`)
161161
- **`tests/shared/`**: Shared test utilities and parametrized cases
162162

163163
**IMPORTANT: `tests/` is for official regression tests ONLY.** Do NOT place throwaway diagnostic,
@@ -255,8 +255,8 @@ The Granite Switch extends the base Granite model with:
255255

256256
2. **Control Tokens**
257257
- Each adapter has a control token `<|adapter|>` that fires the switch
258-
- KV hiding uses group-based control dimensions (K=finfo.min, Q=per-adapter policy)
259-
- Control tokens are KV-hidden to prevent cross-request interference
258+
- The switch reads the original ids, then rewrites each control token to its substitute id
259+
(`adapter_substitute_token_ids`, `token_exchange.py`) so the decoder never embeds it
260260

261261
3. **Chat Template Integration**
262262
- Maps adapter names to control tokens
@@ -267,18 +267,17 @@ The Granite Switch extends the base Granite model with:
267267
"how many control tokens so far" into an address, and a Kerdock/DG coded-memory head
268268
reads back the adapter written there
269269
- Routing is latest-wins and supports any number of transitions per request, including a
270-
return to base via a base-reset control token
270+
return to base via `<|base_reset|>` (only when composed with `--base-reset-token`)
271271
- The codebook is constructed, not trained; config knobs are the `ms_*` fields
272272
- See [docs/MULTISWITCH_EXPLAINED.md](docs/MULTISWITCH_EXPLAINED.md)
273273

274274
### Two Backends
275275

276276
#### HuggingFace Backend (`granite_switch.hf`)
277277

278-
**Purpose**: Model building, prototyping, and training
278+
**Purpose**: Model building, prototyping, and reference inference
279279

280280
- Full `transformers` integration (`PreTrainedModel`, `GenerationMixin`)
281-
- Training with `Trainer` API
282281
- Standard PyTorch operations
283282

284283
#### vLLM Backend (`granite_switch.vllm`)
@@ -308,7 +307,7 @@ set. See [docs/SR_ARCHITECTURE.md](docs/SR_ARCHITECTURE.md) for full details.
308307
Both backends share the same weight format:
309308

310309
```python
311-
# Built/trained with HuggingFace
310+
# Built with HuggingFace
312311
model_hf.save_pretrained("./checkpoint")
313312

314313
# Loaded directly with vLLM
@@ -335,12 +334,12 @@ Always use config values - never hardcode these parameters.
335334
"num_adapters": 4,
336335
"adapter_token_ids": [100, 101, 102, 103],
337336
"adapter_names": ["adapter_0", "adapter_1", "adapter_2", "adapter_3"],
338-
"hiding_groups": {"all_controls": ["adapter_0", "adapter_1", "adapter_2", "adapter_3"]},
339-
"hiding_policy": {"base": ["all_controls"], "adapter_0": ["all_controls"], "...": "..."},
340-
"lora_rank": 8,
341-
"lora_alpha": 8.0,
337+
"adapter_substitute_token_ids": [10, 11, 12, 13],
338+
"max_lora_rank": 8,
339+
"adapter_ranks": [8, 8, 8, 8],
342340
"switch_head_dim": 32,
343-
"control_dims": 32
341+
"ms_code_type": "kerdock",
342+
"ms_code_m": 6
344343
}
345344
```
346345

@@ -394,12 +393,6 @@ attribute on the module.
394393
The HF MultiSwitch tests (`tests/hf/test_multi_switch.py`) probe each non-eager attention backend once
395394
and parametrize over the ones that work on the current platform. Unavailable backends are skipped.
396395

397-
### 7. Known Limitation: Hidden Count Offset When Position 0 is in a Hiding Group
398-
399-
When position 0 is a control token in a hiding group (e.g., a LoRA prefix token with
400-
`add_bos_token=False`), `hidden_count` is off by 1, causing a 1-position RoPE offset. This is
401-
acceptable because adapter detection is exact and RoPE is robust to small positional shifts.
402-
403396
### 8. Known Limitation: TP Row-Parallel Bias Doubling
404397

405398
`SwitchedLoRALinear`'s row-parallel bypass path passes bias to all TP ranks instead of
@@ -446,6 +439,7 @@ runs inside the `@support_torch_compile` region (`src/granite_switch/vllm/granit
446439
default-configured server never sets them. Any test reading them must pass `enforce_eager=True` and
447440
must assert they are present, not skip on `None`. The same reason is why a host-side validation check
448441
cannot live in `MultiSwitch.forward`.
442+
449443
### 12. Pre-Fusion SR Checkpoints Must Be Re-Composed
450444

451445
SR used to build unfused Q/K/V and gate/up/down projections (`unfused_qkv=True`); it now uses
@@ -480,6 +474,9 @@ This repo uses [pre-commit](https://pre-commit.com/) (ruff, hygiene hooks, SPDX-
480474
- `docs/GIT_WORKFLOW.md` - Git branching strategy and commit guidelines
481475
- `docs/SUPPORTED_MODELS.md` - Model compatibility
482476
- `docs/SR_ARCHITECTURE.md` - Shadow Residual dual-stream architecture
477+
- `docs/MULTISWITCH_EXPLAINED.md` - MultiSwitch coded-memory routing engine and its limits
478+
- `docs/AUDIO.md` - Audio input (ASR cascade)
479+
- `docs/CICD.md` - Pre-commit hooks and CI setup
483480

484481
## Git Workflow
485482

‎src/granite_switch/hf/modeling_granite_switch.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -410,7 +410,7 @@ def __init__(self, config: GraniteSwitchConfig):
410410
)
411411

412412
# Token-exchange LUT lives on the switch module (see hf/switch/
413-
# single.py); the switch rewrites input_ids in-place during its
413+
# multi.py); the switch rewrites input_ids in-place during its
414414
# forward pass, so the decoder side needs no substitute table.
415415

416416
else:

‎src/granite_switch/token_exchange.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
Backend-neutral on purpose. The table is a pure function of three config
1010
fields (``vocab_size``, ``adapter_token_ids``, ``adapter_substitute_token_ids``),
1111
so it belongs at the same tier as :mod:`granite_switch.config` rather than
12-
inside either backend — all four switch engines (HF/vLLM x Single/Multi) build
12+
inside either backend — both switch engines (HF and vLLM MultiSwitch) build
1313
the identical table and must agree on its size.
1414
1515
Contract:

0 commit comments

Comments
 (0)