Context
Some GGUF repos include MTP drafter models alongside the main model quants. For example, huihui-ai/Ornith-1.0-9B-abliterated-MTP-GGUF has:
ornith-9b-mtp-kl-BF16.gguf 17.14 GB (main model)
ornith-9b-mtp-kl-Q8_0.gguf 9.11 GB (main model)
ornith-9b-mtp-kl-Q4_K_M.gguf 6.41 GB (main model)
ornith-9b-mtp-kl-IQ3_M.gguf 5.58 GB (main model)
mtp-ornith-9b-mtp-kl-Q8_0.gguf 2.26 GB (MTP drafter)
Similarly, unsloth/gemma-4-E2B-it-GGUF has drafters in an MTP/ subdirectory:
MTP/gemma-4-E2B-it-Q8_0-MTP.gguf 97 MB
MTP/gemma-4-E2B-it-F16-MTP.gguf 170 MB
MTP/gemma-4-E2B-it-BF16-MTP.gguf 170 MB
Current behavior
- The quant picker (in
download.mjs) filters out MTP drafter files (done in v0.18.0)
- Only the selected main model file is downloaded via
hf download <repo> <filename>
- The MTP drafter is NOT downloaded
- The scanner (
scan.mjs) can match drafters to models by filename pattern, but only if the drafter is already on disk
What to build
After the user selects a quant and before the download starts, check if the repo has MTP drafter files. If it does:
- Show a prompt:
This model supports MTP speculative decoding (2x speedup). Also download the drafter model? (Y/n)
- Show the drafter file name and size (e.g.
mtp-ornith-9b-mtp-kl-Q8_0.gguf (2.26 GB))
- If multiple drafter quants exist, pick the smallest one that fits (or let the user choose)
- Download both files: the main model + the drafter
- The scanner will automatically match them when the model is set up
Drafter detection patterns
- Files in an
MTP/ subdirectory: MTP/gemma-4-E2B-it-Q8_0-MTP.gguf
- Files with
mtp- prefix: mtp-ornith-9b-mtp-kl-Q8_0.gguf
- Files with
-MTP. suffix: gemma-4-E2B-it-Q8_0-MTP.gguf
The isDrafterFile() function in huggingface.mjs already detects these patterns. A new function could list ONLY drafter files (the inverse filter).
References
- Drafter matching:
matchDrafter() in scan.mjs
- Drafter detection:
isDrafterFile() in huggingface.mjs
- Quant picker:
pickGgufQuant() in download.mjs
- MTP setup in glass-box flow:
profile-setup.mjs (MTP prompt + drafter detection)
Context
Some GGUF repos include MTP drafter models alongside the main model quants. For example,
huihui-ai/Ornith-1.0-9B-abliterated-MTP-GGUFhas:Similarly,
unsloth/gemma-4-E2B-it-GGUFhas drafters in anMTP/subdirectory:Current behavior
download.mjs) filters out MTP drafter files (done in v0.18.0)hf download <repo> <filename>scan.mjs) can match drafters to models by filename pattern, but only if the drafter is already on diskWhat to build
After the user selects a quant and before the download starts, check if the repo has MTP drafter files. If it does:
This model supports MTP speculative decoding (2x speedup). Also download the drafter model? (Y/n)mtp-ornith-9b-mtp-kl-Q8_0.gguf (2.26 GB))Drafter detection patterns
MTP/subdirectory:MTP/gemma-4-E2B-it-Q8_0-MTP.ggufmtp-prefix:mtp-ornith-9b-mtp-kl-Q8_0.gguf-MTP.suffix:gemma-4-E2B-it-Q8_0-MTP.ggufThe
isDrafterFile()function inhuggingface.mjsalready detects these patterns. A new function could list ONLY drafter files (the inverse filter).References
matchDrafter()inscan.mjsisDrafterFile()inhuggingface.mjspickGgufQuant()indownload.mjsprofile-setup.mjs(MTP prompt + drafter detection)