Promote dev to master: spec-decoding CLI restoration (#34, #37)

MTP.md

@@ -87,7 +87,8 @@ CLI surface (`common/arg.cpp`, `common/common.cpp`):
 
 - `--mtp-head <path>` (preferred) and `--model-draft / -md` (back-compat alias)
   — feed the same `mparams_dft.path` field.
-- `--draft-block-size <B>` — head proposes `B - 1` tokens per round.
+- `--spec-draft-n-max <N>` — head proposes `N` tokens per round (replaces
+  the pre-b10018 `--draft-block-size <B>`, `N = B - 1`).
 - `--gpu-layers-draft / -ngld`, `-ctkd / -ctvd` — placement and KV typing for
   the **assistant** weights when offloaded.
 
@@ -443,7 +444,7 @@ Helper scripts live under `scripts/`:
 
 Edge presets (`MTP_PRESET`):
 
-| Preset | `--draft-block-size` (B) | `--draft-max` |
+| Preset | `DRAFT_BLOCK_SIZE` (B, passed as `--spec-draft-n-max B-1`) | `DRAFT_MAX` (legacy cap, no longer passed) |
 |---|---:|---:|
 | `throughput` | 2 | 6 |
 | `lift`       | 3 | 8 |

NEXTN.md

@@ -39,14 +39,14 @@ the same file in the HF cache and takes the shared-model branch:
 llama-server \
   -hf  AtomicChat/Qwen3.6-35B-A3B-UDT-MTP-GGUF:Q4_K_XL \
   -hfd AtomicChat/Qwen3.6-35B-A3B-UDT-MTP-GGUF:Q4_K_XL \
-  --spec-type nextn --draft-max 2 --draft-min 1 \
+  --spec-type nextn --spec-draft-n-max 2 --spec-draft-n-min 1 \
   -c 8192 -ngl 99 -ngld 99 -fa on
 
 # 27B dense
 llama-server \
   -hf  AtomicChat/Qwen3.6-27B-UDT-MTP-GGUF:Q4_K_XL \
   -hfd AtomicChat/Qwen3.6-27B-UDT-MTP-GGUF:Q4_K_XL \
-  --spec-type nextn --draft-max 2 --draft-min 1 \
+  --spec-type nextn --spec-draft-n-max 2 --spec-draft-n-min 1 \
   -c 8192 -ngl 99 -ngld 99 -fa on
 ```
 
@@ -76,7 +76,7 @@ that used to OOM the 35B-A3B target on Apple Silicon (38 GB unified memory). See
   and switches to the shared-model path (no second model load). Pointing at a standalone
   NEXTN_ONLY GGUF (`general.architecture = qwen35*_mtp`) still works but loads a second
   `llama_model`.
-- `--draft-max` / `--spec-draft-n-max` — max chained draft tokens per round (see `common` / server arg naming).
+- `--spec-draft-n-max` — max chained draft tokens per round (pre-b10018 name: `--draft-max`).
 - Gemma MTP flags (`--mtp-head`, `llama_decode_mtp_*`, `llama_model_load_mtp_from_file`) are **unchanged**.
 
 ---
@@ -123,14 +123,14 @@ let `llama-server` pull both from Hugging Face into the local cache:
 llama-server \
   -hf  AtomicChat/Qwen3.6-35B-A3B-UDT-MTP-GGUF:Q4_K_XL \
   -hfd AtomicChat/Qwen3.6-35B-A3B-UDT-MTP-GGUF:Q4_K_XL \
-  --spec-type nextn --draft-max 2 --draft-min 1
+  --spec-type nextn --spec-draft-n-max 2 --spec-draft-n-min 1
 ```
 
 ---
 
 ## 7. Performance notes (MacBook Pro M4 Max, 40-core GPU, 48 GB, Metal)
 
-Median TPS over 2 runs, prompt = 50-token instruction, `--draft-max=2 --draft-min=1`,
+Median TPS over 2 runs, prompt = 50-token instruction, `--spec-draft-n-max=2 --spec-draft-n-min=1`,
 NextN draft DM=2 (single async chain), context 8192. Single-slot
 (`--parallel 1 -np 1 --cont-batching`), full GPU offload (`-ngl 99 -ngld 99 -fa on`),
 shared-model draft path (no second mmap of combined `_MTP.gguf`),

README.md

@@ -117,7 +117,7 @@ llama-server \
   -m /path/to/gemma-4-target.gguf \
   --mtp-head /path/to/gemma-4-assistant-Q4_K_M.gguf \
   --spec-type mtp \
-  --draft-block-size 3 \
+  --spec-draft-n-max 2 \
   -c 16384 \
   -ngl 99 -ngld 99 \
   -fa on \
@@ -140,8 +140,8 @@ MTP_PRESET=throughput scripts/run-gemma4-e2b-mtp-server.sh
 ### Bench snapshot (MacBook Pro M4 Max, 40-core GPU, 48 GB, Metal, single slot)
 
 Median tps over 3 runs with Q4_K_M assistant heads. Dense scripts default to
-`--draft-block-size 3`; E4B uses `MTP_PRESET=throughput` (`B = 2`,
-`--draft-max 6`). See
+two drafted tokens per round (`--spec-draft-n-max 2`); E4B uses
+`MTP_PRESET=throughput` (one drafted token per round). See
 `.scratch/bench-logs/gemma-matrix-fullrun-20260512-224705.md`.
 
 | model | mode | n=128 tps | n=512 tps | accept@128 | accept@512 |
@@ -158,8 +158,8 @@ Median tps over 3 runs with Q4_K_M assistant heads. Dense scripts default to
 
 ### Knobs
 
-- `--draft-block-size B` — head emits `B - 1` tokens per round (default 4;
-  bench used 3).
+- `--spec-draft-n-max N` — head emits `N` tokens per round (bench used 2;
+  replaces the pre-b10018 `--draft-block-size B`, `N = B - 1`).
 - `--mtp-head <path>` (preferred) / `-md <path>` (back-compat alias).
 - `LLAMA_MTP_SKIP_STREAK_THRESHOLD=N` — adaptive skip after `N` consecutive
   zero-accept batches (off by default).
@@ -225,7 +225,7 @@ llama-server \
   -hf  AtomicChat/Qwen3.6-35B-A3B-UDT-MTP-GGUF:Q4_K_XL \
   -hfd AtomicChat/Qwen3.6-35B-A3B-UDT-MTP-GGUF:Q4_K_XL \
   --spec-type nextn \
-  --draft-max 2 --draft-min 1 \
+  --spec-draft-n-max 2 --spec-draft-n-min 1 \
   -c 8192 \
   -ngl 99 -ngld 99 \
   -ctk turbo3 -ctv turbo3 -fa on \
@@ -238,7 +238,7 @@ Or with a local file (e.g. the artifact stored under `.scratch/`):
 llama-server \
   -m   /path/to/Qwen3.6-35B-A3B-UD-Q4_K_XL_MTP.gguf \
   -md  /path/to/Qwen3.6-35B-A3B-UD-Q4_K_XL_MTP.gguf \
-  --spec-type nextn --draft-max 2 --draft-min 1 \
+  --spec-type nextn --spec-draft-n-max 2 --spec-draft-n-min 1 \
   -c 8192 -ngl 99 -ngld 99 -ctk turbo3 -ctv turbo3 -fa on
 ```
 
@@ -256,7 +256,7 @@ If you ship the NextN head as a separate **NEXTN_ONLY** GGUF
 
 ### Bench snapshot (MacBook Pro M4 Max, 40-core GPU, 48 GB, Metal, single slot)
 
-Median tps over 3 runs, `--draft-max 2 --draft-min 1`, single-slot, shared
+Median tps over 3 runs, `--spec-draft-n-max 2 --spec-draft-n-min 1`, single-slot, shared
 target/draft model. See
 `.scratch/bench-logs/qwen-matrix-fullrun-20260512-222625.md`.
 
@@ -277,7 +277,7 @@ target/draft model. See
 - `--model-draft` / `-md` — pass the **same** path as `--model` for the
   shared-model path; pass a NEXTN_ONLY GGUF to use the legacy double-load
   fallback.
-- `--draft-max` / `--draft-min` — chained-draft bounds per round
+- `--spec-draft-n-max` / `--spec-draft-n-min` — chained-draft bounds per round
   (current default for the helper scripts: `2 / 1`).
 - `llama_set_nextn` (C API) — pairs target and draft contexts so that
   `llama_context_nextn_seq_rm` trims **both** KV caches in one call.
@@ -750,7 +750,7 @@ To learn more about model quantization, [read this documentation](tools/quantize
       -m   /path/to/gemma-4-target.gguf \
       --mtp-head /path/to/gemma-4-assistant-Q4_K_M.gguf \
       --spec-type mtp \
-      --draft-block-size 3 \
+      --spec-draft-n-max 2 \
       -c 16384 \
       -ngl 99 -ngld 99 \
       -fa on \
@@ -808,7 +808,7 @@ To learn more about model quantization, [read this documentation](tools/quantize
       -hf  unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \
       -hfd unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \
       --spec-type nextn \
-      --draft-max 2 --draft-min 1 \
+      --spec-draft-n-max 2 --spec-draft-n-min 1 \
       -c 8192 \
       -ngl 99 -ngld 99 \
       -ctk turbo3 -ctv turbo3 -fa on \
@@ -821,7 +821,7 @@ To learn more about model quantization, [read this documentation](tools/quantize
     llama-server \
       -m   /path/to/Qwen3.6-35B-A3B-UD-Q4_K_XL_MTP.gguf \
       -md  /path/to/Qwen3.6-35B-A3B-UD-Q4_K_XL_MTP.gguf \
-      --spec-type nextn --draft-max 2 --draft-min 1 \
+      --spec-type nextn --spec-draft-n-max 2 --spec-draft-n-min 1 \
       -c 8192 -ngl 99 -ngld 99 -ctk turbo3 -ctv turbo3 -fa on
     ```
 

common/arg.cpp

@@ -3877,8 +3877,8 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
         }
     ).set_spec().set_examples({LLAMA_EXAMPLE_SPECULATIVE, LLAMA_EXAMPLE_SERVER, LLAMA_EXAMPLE_CLI}).set_env("LLAMA_ARG_N_GPU_LAYERS_DRAFT"));
     add_opt(common_arg(
-        {"--spec-draft-model", "-md", "--model-draft"}, "FNAME",
-        "draft model for speculative decoding (default: unused)",
+        {"--spec-draft-model", "-md", "--model-draft", "--mtp-head"}, "FNAME",
+        "draft model for speculative decoding, also accepts an MTP/NextN head GGUF (default: unused)",
         [](common_params & params, const std::string & value) {
             params.speculative.draft.mparams.path = value;
             params.speculative.draft.mparams.hf_file = value; // will be used if --spec-draft-hf is set

common/speculative.cpp

@@ -38,7 +38,12 @@ const std::map<std::string, common_speculative_type> common_speculative_type_fro
     {"ngram-map-k",   COMMON_SPECULATIVE_TYPE_NGRAM_MAP_K},
     {"ngram-map-k4v", COMMON_SPECULATIVE_TYPE_NGRAM_MAP_K4V},
     {"ngram-mod",     COMMON_SPECULATIVE_TYPE_NGRAM_MOD},
-    {"ngram-cache",   COMMON_SPECULATIVE_TYPE_NGRAM_CACHE}
+    {"ngram-cache",   COMMON_SPECULATIVE_TYPE_NGRAM_CACHE},
+    // fork aliases: Gemma 4 MTP ("mtp", MTP.md) and Qwen NextN ("nextn",
+    // NEXTN.md) both route to the draft-mtp implementation since the b10018
+    // upstream sync. Kept so commands from the fork docs keep working.
+    {"mtp",           COMMON_SPECULATIVE_TYPE_DRAFT_MTP},
+    {"nextn",         COMMON_SPECULATIVE_TYPE_DRAFT_MTP}
 };
 
 static std::string common_speculative_get_devices_str(const std::vector<ggml_backend_dev_t> & devices) {

docs/speculative.md

@@ -31,7 +31,7 @@ A draft model is the most used approach in speculative decoding.
 For **Gemma 4** targets with the **gemma4_assistant** (MTP head) GGUF, use `--spec-type mtp`. The assistant is **not** a second `llama_context`: weights are loaded into the target model via `llama_model_load_mtp_from_file` (done automatically when using the server/CLI init path). Cross-attention in the MTP graph reads **K/V from the target KV cache** (shared full/sliding layers).
 
 - Prefer **`--mtp-head /path/to/assistant.gguf`** for clarity; **`--model-draft` (`-md`)** is accepted as a backward-compatible alias (same path field).
-- The draft block size \(B\) is **`--draft-block-size`** (the head proposes `B - 1` tokens per round; default 4).
+- The per-round draft length is **`--spec-draft-n-max N`** (the head proposes `N` tokens per round; replaces the pre-b10018 `--draft-block-size B`, `N = B - 1`).
 - **`--gpu-layers-draft` / `-ngld`** and **`-ctkd` / `-ctvd`** still apply to how the **assistant tensors** are placed and typed when the assistant GGUF is loaded; the target uses `-ngl` and `-ctk`/`-ctv`.
 
 Example (paths illustrative). **TurboQuant** KV on the target: `-ctk`/`-ctv`. Assistant-side cache types follow the draft flags if you use them for offload/quant selection.
@@ -41,7 +41,7 @@ llama-server \
   -m /path/to/gemma-4-target.gguf \
   --mtp-head /path/to/gemma-4-assistant.gguf \
   --spec-type mtp \
-  --draft-block-size 4 \
+  --spec-draft-n-max 3 \
   -c 16384 \
   -ngl 99 -ngld 99 \
   -ctk turbo3 -ctv turbo3 \
@@ -89,7 +89,7 @@ Why use the async pair?
 
 - **Graph-cache isolation**: MTP graph reuse is no longer invalidated by target
   decode resets. On Gemma 4 + Q4_K_XL, this alone delivered **~+8% throughput**
-  in single-slot benchmarks (95.3 → 102.8 tps at `--draft-block-size 3`), with
+  in single-slot benchmarks (95.3 → 102.8 tps at `--spec-draft-n-max 2`), with
   identical accept rate.
 - **Pipeline depth-2 (pure overlap, `llama-server`)**: after target
   sample/accept and `llama_memory_seq_rm`, the server calls
@@ -166,7 +166,7 @@ llama-server \
   -m /path/to/qwen3.6-MTP.gguf \
   -md /path/to/qwen3.6-MTP.gguf \
   --spec-type nextn \
-  --draft-max 2 --draft-min 1 \
+  --spec-draft-n-max 2 --spec-draft-n-min 1 \
   -c 8192 -ngl 99 -ngld 99 -fa on \
   --host 127.0.0.1 --port 8080
 ```

gguf-py/gguf/tensor_mapping.py

@@ -2415,18 +2415,6 @@ class TensorNameMap:
             "model.layers.{bid}.shared_head.norm",
         ),
 
-        MODEL_TENSOR.MTP_PRE_PROJECTION: (
-            "pre_projection",
-        ),
-        MODEL_TENSOR.MTP_POST_PROJECTION: (
-            "post_projection",
-        ),
-        MODEL_TENSOR.MTP_CENTROIDS: (
-            "masked_embedding.centroids",
-        ),
-        MODEL_TENSOR.MTP_TOKEN_ORDERING: (
-            "masked_embedding.token_ordering",
-        ),
     }
 
     # architecture-specific block mappings

scripts/run-gemma4-31b-mtp-server.sh

@@ -86,9 +86,10 @@ if [[ "$SPEC" == "mtp" ]]; then
   ARGS+=(
     --mtp-head "$DRAFT"
     --spec-type mtp
-    --draft-block-size "${DRAFT_BLOCK_SIZE:-3}"
-    --draft-max "${DRAFT_MAX:-16}"
-    --draft-min "${DRAFT_MIN:-0}"
+    # b10018 sync: --draft-block-size/--draft-max are gone; per-round chain
+    # length is --spec-draft-n-max (= old block size B - 1).
+    --spec-draft-n-max "$(( ${DRAFT_BLOCK_SIZE:-3} - 1 ))"
+    --spec-draft-n-min "${DRAFT_MIN:-0}"
   )
 else
   echo "info: speculative decoding disabled (SPEC=${SPEC}); running baseline" >&2

scripts/run-gemma4-e2b-mtp-server.sh

@@ -111,9 +111,10 @@ if [[ "$SPEC" == "mtp" ]]; then
   ARGS+=(
     --mtp-head "$DRAFT"
     --spec-type mtp
-    --draft-block-size "${DRAFT_BLOCK_SIZE}"
-    --draft-max "${DRAFT_MAX}"
-    --draft-min "${DRAFT_MIN:-0}"
+    # b10018 sync: --draft-block-size/--draft-max are gone; per-round chain
+    # length is --spec-draft-n-max (= old block size B - 1).
+    --spec-draft-n-max "$(( ${DRAFT_BLOCK_SIZE} - 1 ))"
+    --spec-draft-n-min "${DRAFT_MIN:-0}"
   )
 else
   echo "info: speculative decoding disabled (SPEC=${SPEC}); running baseline" >&2

scripts/run-gemma4-e4b-mtp-server.sh

@@ -117,9 +117,10 @@ if [[ "$SPEC" == "mtp" ]]; then
   ARGS+=(
     --mtp-head "$DRAFT"
     --spec-type mtp
-    --draft-block-size "${DRAFT_BLOCK_SIZE}"
-    --draft-max "${DRAFT_MAX}"
-    --draft-min "${DRAFT_MIN:-0}"
+    # b10018 sync: --draft-block-size/--draft-max are gone; per-round chain
+    # length is --spec-draft-n-max (= old block size B - 1).
+    --spec-draft-n-max "$(( ${DRAFT_BLOCK_SIZE} - 1 ))"
+    --spec-draft-n-min "${DRAFT_MIN:-0}"
   )
 else
   echo "info: speculative decoding disabled (SPEC=${SPEC}); running baseline" >&2

scripts/run-gemma4-mtp-server.sh

@@ -87,9 +87,10 @@ if [[ "$SPEC" == "mtp" ]]; then
   ARGS+=(
     --mtp-head "$DRAFT"
     --spec-type mtp
-    --draft-block-size "${DRAFT_BLOCK_SIZE:-3}"
-    --draft-max "${DRAFT_MAX:-16}"
-    --draft-min "${DRAFT_MIN:-0}"
+    # b10018 sync: --draft-block-size/--draft-max are gone; per-round chain
+    # length is --spec-draft-n-max (= old block size B - 1).
+    --spec-draft-n-max "$(( ${DRAFT_BLOCK_SIZE:-3} - 1 ))"
+    --spec-draft-n-min "${DRAFT_MIN:-0}"
   )
 else
   echo "info: speculative decoding disabled (SPEC=${SPEC}); running baseline" >&2

scripts/run-gemma4.sh

@@ -108,9 +108,10 @@ if [[ "$SPEC" == "mtp" ]]; then
     -ctvd "$CTVD"
     --mtp-head "$DRAFT"
     --spec-type mtp
-    --draft-block-size "${DRAFT_BLOCK_SIZE:-4}"
-    --draft-max "${DRAFT_MAX:-16}"
-    --draft-min "${DRAFT_MIN:-0}"
+    # b10018 sync: --draft-block-size/--draft-max are gone; per-round chain
+    # length is --spec-draft-n-max (= old block size B - 1).
+    --spec-draft-n-max "$(( ${DRAFT_BLOCK_SIZE:-4} - 1 ))"
+    --spec-draft-n-min "${DRAFT_MIN:-0}"
   )
 fi
 

scripts/run-qwen36-27b-nextn-server.sh

@@ -69,8 +69,8 @@ if [[ "$SPEC" == "nextn" ]]; then
   ARGS+=(
     -md "$DRAFT"
     --spec-type nextn
-    --draft-max "${DRAFT_MAX:-16}"
-    --draft-min "${DRAFT_MIN:-0}"
+    --spec-draft-n-max "${DRAFT_MAX:-2}"
+    --spec-draft-n-min "${DRAFT_MIN:-1}"
   )
 else
   echo "info: speculative decoding disabled (SPEC=${SPEC}); running baseline" >&2

src/llama-model.cpp

@@ -330,6 +330,13 @@ llama_model * llama_model_create(llm_arch arch, const llama_model_params & param
 llama_model * llama_model_create(llama_model_loader & ml, const llama_model_params & params) {
     llm_arch arch = ml.get_arch();
     if (arch == LLM_ARCH_UNKNOWN) {
+        if (ml.get_arch_name() == "gemma4_assistant") {
+            // pre-b10018 fork assistant GGUFs use a different arch name, kv
+            // namespace and tensor layout than the upstream gemma4-assistant
+            // support; they cannot be loaded by this build.
+            throw std::runtime_error("this Gemma 4 assistant GGUF predates the b10018 sync and is no longer supported — "
+                                     "re-download the regenerated assistant or reconvert it with conversion/gemma.py");
+        }
         throw std::runtime_error("unknown model architecture: '" + ml.get_arch_name() + "'");
     }