Skip to content

Commit 86bc033

Browse files
authored
docs: update CLAUDE.md and AUDIO.md to match current code (#143)
* docs: update CLAUDE.md and AUDIO.md to match code CLAUDE.md still described the removed SingleSwitch: the project tree listed hf/switch/single.py and vllm/switch/single.py, the import example and test commands pointed at SingleSwitch files that no longer exist, and the architecture section described a trainable router. It now reflects the coded-memory MultiSwitch (the only engine since #131), links docs/MULTISWITCH_EXPLAINED.md, and lists conversation.py, token_exchange.py, hf/switch/codes/, vllm/audio/ and tests/audio/. The dev install command used `--extra dev`, but `dev` is a dependency group, so it is now `--group dev`; the nonexistent tests/regression/ entry is dropped. docs/AUDIO.md said transformers>=5.16 was not pinned yet. The package has required it since #130, so the caveat now says a normal install already has it and keeps the ImportError fallback note for older installs. Signed-off-by: samadarsh <samadarsh14@gmail.com> * docs: drop leftover router references in CLAUDE.md The overview still listed "optional trainable switching" with a router, the adapter section said adapters are "router-selected", and the test-directory list still named tests/regression/, which does not exist. MultiSwitch has no trainable router: it derives the per-token adapter index from the control tokens. The test list now names tests/audio/ instead. Signed-off-by: samadarsh <samadarsh14@gmail.com> --------- Signed-off-by: samadarsh <samadarsh14@gmail.com>
1 parent 9c0c3ea commit 86bc033

2 files changed

Lines changed: 33 additions & 25 deletions

File tree

‎CLAUDE.md‎

Lines changed: 28 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
99
1. **Building models with embedded adapters** - Combine a base Granite model with multiple LoRA adapters into a single checkpoint
1010
2. **Automatic adapter control** - Activate adapters via special control tokens or chat templates
1111
3. **Fast inference** - Deploy with vLLM for speedup over standard HuggingFace inference
12-
4. **Optional trainable switching** - Train a router to automatically select adapters per-token
12+
4. **Multi-transition switching** - MultiSwitch picks the adapter per token from the control tokens seen so far (latest wins), so one request can switch adapters any number of times
1313

1414
## Project Structure
1515

@@ -20,6 +20,8 @@ granite-switch/
2020
│ └── granite_switch/ # Unified package
2121
│ ├── __init__.py # Core exports (GraniteSwitchConfig, __version__)
2222
│ ├── config.py # Unified GraniteSwitchConfig
23+
│ ├── conversation.py # Conversation: multi-turn, per-turn adapter, KV-history policy
24+
│ ├── token_exchange.py # Control-token -> substitute-id LUT (shared by both backends)
2325
│ │
2426
│ ├── composer/ # Compose system (requires [compose] extra)
2527
│ │ ├── __init__.py
@@ -47,8 +49,9 @@ granite-switch/
4749
│ │ │ ├── __init__.py
4850
│ │ │ └── lora.py # SwitchedLoRALinear, MergedSwitchedLoRALinear
4951
│ │ └── switch/
50-
│ │ ├── __init__.py
51-
│ │ └── single.py # SingleSwitch (HF attention backends)
52+
│ │ ├── __init__.py # create_switch() -> MultiSwitch
53+
│ │ ├── multi.py # MultiSwitch (HF attention backends)
54+
│ │ └── codes/ # Kerdock/DG codebook generation for the memory head
5255
│ │
5356
│ ├── kernels/ # Backend-agnostic Triton kernels
5457
│ │ ├── __init__.py
@@ -71,17 +74,18 @@ granite-switch/
7174
│ │ ├── wcross_shunt.py # WCrossShunt (base->adapter cross-stream)
7275
│ │ ├── kernel_meta.py # SRFusedLoRAKernelMeta
7376
│ │ └── _sr_ops.py # Q-head interleave / deinterleave
77+
│ ├── audio/ # Audio input (alpha): ASR cascade, chunking, processor
7478
│ └── switch/
75-
│ ├── __init__.py
76-
│ └── single.py # SingleSwitch (vLLM Attention)
79+
│ ├── __init__.py # create_switch() -> MultiSwitch
80+
│ └── multi.py # MultiSwitch (vLLM Attention)
7781
│
7882
├── tests/ # All tests
7983
│ ├── unit/ # Unit tests (fastest, CPU)
8084
│ ├── hf/ # HuggingFace-specific tests
8185
│ ├── vllm/ # vLLM-specific tests
8286
│ ├── composer/ # Compose system tests
8387
│ ├── integration/ # Cross-backend integration tests
84-
│ ├── regression/ # Regression tests (hf/, vllm/, integration/, shared/, tools/)
88+
│ ├── audio/ # Audio/ASR tests (pytest -m audio)
8589
│ └── shared/ # Shared test utilities and parametrized cases
8690
│
8791
├── .pre-commit/ # Pre-commit hook scripts (validate_links.py)
@@ -110,8 +114,8 @@ uv sync --extra vllm
110114
# With compose tools
111115
uv sync --extra compose
112116

113-
# Everything (development)
114-
uv sync --extra dev
117+
# Everything (development) -- `dev` is a dependency group, not an extra
118+
uv sync --group dev
115119
```
116120

117121
## Import Paths
@@ -124,7 +128,7 @@ from granite_switch.config import GraniteSwitchConfig # equivalent
124128
# HuggingFace backend
125129
from granite_switch.hf import GraniteSwitchForCausalLM
126130
from granite_switch.hf.core.lora import SwitchedLoRALinear
127-
from granite_switch.hf.switch.single import SingleSwitch
131+
from granite_switch.hf.switch import MultiSwitch, create_switch
128132

129133
# vLLM backend (auto-registered via plugin entry point)
130134
from granite_switch.vllm import register
@@ -153,7 +157,7 @@ from granite_switch.composer import GraniteSwitchComposer
153157
- **`tests/vllm/`**: vLLM implementation tests
154158
- **`tests/composer/`**: Compose system tests
155159
- **`tests/integration/`**: Cross-implementation and end-to-end integration tests
156-
- **`tests/regression/`**: Regression tests (hf/, vllm/, integration/, shared/, tools/)
160+
- **`tests/audio/`**: Audio/ASR tests (`pytest -m audio`)
157161
- **`tests/shared/`**: Shared test utilities and parametrized cases
158162

159163
**IMPORTANT: `tests/` is for official regression tests ONLY.** Do NOT place throwaway diagnostic,
@@ -209,11 +213,11 @@ single command:
209213
pytest tests/unit/ -v -s --tb=short -x
210214

211215
# 2. HF tests by file (CPU)
212-
pytest tests/hf/test_single_switch.py -v -s --tb=short -x
216+
pytest tests/hf/test_multi_switch.py -v -s --tb=short -x
213217
pytest tests/hf/test_model_forward.py -v -s --tb=short -x
214218

215219
# 3. vLLM tests by file (GPU required)
216-
pytest tests/vllm/test_single_switch.py -v -s --tb=short -x
220+
pytest tests/vllm/test_multi_switch.py -v -s --tb=short -x
217221
pytest tests/vllm/test_model_forward.py -v -s --tb=short -x
218222

219223
# 4. Integration tests last (slowest, GPU required)
@@ -247,7 +251,7 @@ The Granite Switch extends the base Granite model with:
247251
1. **Embedded LoRA Adapters** (frozen during inference)
248252
- Multiple task/domain-specific adapters embedded in the same checkpoint
249253
- Each adapter has LoRA weights (lora_A, lora_B) stacked in tensors
250-
- Controlled via special tokens or router-selected indices
254+
- Controlled via control tokens; MultiSwitch turns them into a per-token adapter index
251255

252256
2. **Control Tokens**
253257
- Each adapter has a control token `<|adapter|>` that fires the switch
@@ -258,16 +262,20 @@ The Granite Switch extends the base Granite model with:
258262
- Maps adapter names to control tokens
259263
- Automatic token placement based on adapter type (ALORA vs LORA)
260264

261-
4. **Optional Trainable Router** (SingleSwitch)
262-
- N transformer layers that compute adapter indices per-token
263-
- Linear projection head to num_adapters dimensions
264-
- ~1-2% of total model parameters
265+
4. **MultiSwitch** (the only switch engine; SingleSwitch was removed)
266+
- Two small attention heads compute the adapter index per token: a counting head turns
267+
"how many control tokens so far" into an address, and a Kerdock/DG coded-memory head
268+
reads back the adapter written there
269+
- Routing is latest-wins and supports any number of transitions per request, including a
270+
return to base via a base-reset control token
271+
- The codebook is constructed, not trained; config knobs are the `ms_*` fields
272+
- See [docs/MULTISWITCH_EXPLAINED.md](docs/MULTISWITCH_EXPLAINED.md)
265273

266274
### Two Backends
267275

268276
#### HuggingFace Backend (`granite_switch.hf`)
269277

270-
**Purpose**: Model building and optional router training
278+
**Purpose**: Model building, prototyping, and training
271279

272280
- Full `transformers` integration (`PreTrainedModel`, `GenerationMixin`)
273281
- Training with `Trainer` API
@@ -383,8 +391,8 @@ The eager backend does NOT handle `attention_mask=None` as causal — it treats
383391
(full attention). SDPA and FlashAttention handle `attention_mask=None` correctly via `is_causal`
384392
attribute on the module.
385393

386-
The HF stress tests (`tests/hf/test_single_switch.py`) auto-detect which attention backends work on the
387-
current platform by probing each with a k=-inf GQA call at import time. Unavailable backends are skipped.
394+
The HF MultiSwitch tests (`tests/hf/test_multi_switch.py`) probe each non-eager attention backend once
395+
and parametrize over the ones that work on the current platform. Unavailable backends are skipped.
388396

389397
### 7. Known Limitation: Hidden Count Offset When Position 0 is in a Hiding Group
390398

‎docs/AUDIO.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@ extra, which the `audio` extra here pulls in (as `vllm[audio]`). A plain
1818
non-16 kHz input.
1919

2020
The default ASR model needs **transformers >= 5.16**, the release that added
21-
`granite_speech5_ctc` — its architecture. **That version is not pinned yet**: the
22-
package currently caps transformers below it, and raising the cap is tracked
23-
separately (issue #84). Until that lands, the first transcription on the default
24-
model raises an `ImportError` naming the fix, and the cascade works today only
25-
with an `--asr-model` your installed transformers supports (for example
21+
`granite_speech5_ctc` — its architecture. The package itself requires
22+
`transformers>=5.16` (see `pyproject.toml`), so a normal install already has it.
23+
If an older transformers ends up installed anyway, the first transcription on the
24+
default model raises an `ImportError` naming the fix; you can also point the
25+
checkpoint at any `--asr-model` your installed transformers supports (for example
2626
`openai/whisper-small`).
2727

2828
```bash

0 commit comments

Comments
 (0)