FrontendAPI] ERROR Backend supervisor: KeyError: 'model.layers.0.mlp.shared_expert.gate_up_proj.weight_scale'

[XeonG]

[13:20:02] stdout/INFO  checkpoint started (id=Ornith-1.5-35B-A3B-uncensored-NVFP4 pid=14684)
[13:20:05] stdout/INFO  Converting dense weights: 0it [00:00, ?it/s]
[13:20:06] stdout/INFO  Loading weights:   0%|          | 0/3 [00:00<?, ?it/s]
[13:20:07] stdout/INFO  Converting dense weights: 161it [00:01, 170.23it/s]
[13:20:07] stdout/INFO  
[13:20:07] stdout/INFO  Loading weights:  33%|███▎      | 1/3 [00:01<00:03,  1.90s/it]
[13:20:08] stdout/INFO  Converting dense weights: 397it [00:03, 217.22it/s]
[13:20:08] stdout/INFO  
[13:20:09] stdout/INFO  Loading weights:  67%|██████▋   | 2/3 [00:03<00:01,  1.50s/it]
[13:20:09] stdout/INFO  Converting dense weights: 447it [00:04, 99.85it/s]
[13:20:09] stdout/INFO  
[13:20:09] stdout/INFO  Loading weights: 100%|██████████| 3/3 [00:04<00:00,  1.35s/it]
[13:20:09] stdout/INFO  [2026-08-29|13:20:09|core|rank=0] INFO     expert banks: slow path (serial build)
[13:20:10] stdout/INFO  Loading Qwen3.5 NVFP4 experts:   0%|          | 0/3 [00:00<?, ?it/s]
[13:20:10] stdout/INFO  Converting expert banks: 0.00B [00:00, ?B/s]
[13:20:10] stdout/INFO  Converting expert banks: 434MB [00:00, 1.47GB/s]
[13:20:11] stdout/INFO  
[13:20:11] stdout/INFO  Converting expert banks: 867MB [00:00, 876MB/s]
[13:20:12] stdout/INFO  
[13:20:12] stdout/INFO  Converting expert banks: 1.27GB [00:01, 761MB/s]
[13:20:12] stdout/INFO  
[13:20:12] stdout/INFO  Converting expert banks: 1.69GB [00:02, 724MB/s]
[13:20:13] stdout/INFO  
[13:20:13] stdout/INFO  Converting expert banks: 2.12GB [00:03, 695MB/s]
[13:20:14] stdout/INFO  
[13:20:14] stdout/INFO  Converting expert banks: 2.54GB [00:03, 661MB/s]
[13:20:14] stdout/INFO  
[13:20:14] stdout/INFO  Converting expert banks: 2.96GB [00:04, 641MB/s]
[13:20:15] stdout/INFO  
[13:20:15] stdout/INFO  Converting expert banks: 3.39GB [00:05, 607MB/s]
[13:20:16] stdout/INFO  
[13:20:16] stdout/INFO  Converting expert banks: 3.81GB [00:06, 619MB/s]
[13:20:17] stdout/INFO  
[13:20:17] stdout/INFO  Converting expert banks: 4.23GB [00:06, 629MB/s]
[13:20:17] stdout/INFO  
[13:20:17] stdout/INFO  Converting expert banks: 4.66GB [00:07, 634MB/s]
[13:20:18] stdout/INFO  
[13:20:18] stdout/INFO  Converting expert banks: 5.08GB [00:08, 646MB/s]
[13:20:19] stdout/INFO  
[13:20:19] stdout/INFO  Converting expert banks: 5.50GB [00:08, 646MB/s]
[13:20:19] stdout/INFO  
[13:20:19] stdout/INFO  Converting expert banks: 5.93GB [00:09, 654MB/s]
[13:20:20] stdout/INFO  
[13:20:20] stdout/INFO  Converting expert banks: 6.35GB [00:10, 660MB/s]
[13:20:21] stdout/INFO  Loading Qwen3.5 NVFP4 experts:  33%|███▎      | 1/3 [00:10<00:21, 10.63s/it]
[13:20:21] stdout/INFO  Converting expert banks: 6.77GB [00:11, 569MB/s]
[13:20:22] stdout/INFO  
[13:20:22] stdout/INFO  Converting expert banks: 7.20GB [00:11, 592MB/s]
[13:20:23] stdout/INFO  
[13:20:23] stdout/INFO  Converting expert banks: 7.62GB [00:12, 608MB/s]
[13:20:23] stdout/INFO  
[13:20:23] stdout/INFO  Converting expert banks: 8.04GB [00:13, 625MB/s]
[13:20:24] stdout/INFO  
[13:20:24] stdout/INFO  Converting expert banks: 8.47GB [00:14, 630MB/s]
[13:20:25] stdout/INFO  
[13:20:25] stdout/INFO  Converting expert banks: 8.89GB [00:14, 640MB/s]
[13:20:25] stdout/INFO  
[13:20:25] stdout/INFO  Converting expert banks: 9.31GB [00:15, 644MB/s]
[13:20:26] stdout/INFO  
[13:20:26] stdout/INFO  Converting expert banks: 9.74GB [00:16, 644MB/s]
[13:20:27] stdout/INFO  
[13:20:27] stdout/INFO  Converting expert banks: 10.2GB [00:16, 627MB/s]
[13:20:28] stdout/INFO  
[13:20:28] stdout/INFO  Converting expert banks: 10.6GB [00:17, 578MB/s]
[13:20:29] stdout/INFO  
[13:20:29] stdout/INFO  Converting expert banks: 11.0GB [00:18, 600MB/s]
[13:20:29] stdout/INFO  
[13:20:29] stdout/INFO  Converting expert banks: 11.4GB [00:19, 577MB/s]
[13:20:30] stdout/INFO  
[13:20:30] stdout/INFO  Converting expert banks: 11.9GB [00:20, 599MB/s]
[13:20:31] stdout/INFO  
[13:20:31] stdout/INFO  Converting expert banks: 12.3GB [00:20, 577MB/s]
[13:20:32] stdout/INFO  
[13:20:32] stdout/INFO  Converting expert banks: 12.7GB [00:21, 596MB/s]
[13:20:32] stdout/INFO  
[13:20:32] stdout/INFO  Converting expert banks: 13.1GB [00:22, 576MB/s]
[13:20:33] stdout/INFO  
[13:20:33] stdout/INFO  Converting expert banks: 13.5GB [00:23, 591MB/s]
[13:20:34] stdout/INFO  
[13:20:34] stdout/INFO  Converting expert banks: 14.0GB [00:23, 604MB/s]
[13:20:35] stdout/INFO  
[13:20:35] stdout/INFO  Converting expert banks: 14.4GB [00:24, 620MB/s]
[13:20:36] stdout/INFO  Loading Qwen3.5 NVFP4 experts:  67%|██████▋   | 2/3 [00:25<00:12, 12.97s/it]
[13:20:36] stdout/INFO  Converting expert banks: 14.8GB [00:26, 473MB/s]
[13:20:37] stdout/INFO  
[13:20:37] stdout/INFO  Converting expert banks: 15.2GB [00:26, 518MB/s]
[13:20:38] stdout/INFO  
[13:20:38] stdout/INFO  Converting expert banks: 15.7GB [00:27, 537MB/s]
[13:20:38] stdout/INFO  
[13:20:38] stdout/INFO  Converting expert banks: 16.1GB [00:28, 545MB/s]
[13:20:39] stdout/INFO  
[13:20:39] stdout/INFO  Converting expert banks: 16.5GB [00:29, 570MB/s]
[13:20:40] stdout/INFO  
[13:20:40] stdout/INFO  Converting expert banks: 16.9GB [00:29, 566MB/s]
[13:20:40] stdout/INFO  Loading Qwen3.5 NVFP4 experts: 100%|██████████| 3/3 [00:30<00:00, 10.10s/it]
[13:20:40] stdout/INFO  [2026-08-29|13:20:40|core|rank=0] INFO     NVFP4 expert backend: triton
[13:20:40] stdout/INFO    (×2)
[13:20:40] stdout/INFO  wrote FTW checkpoint -> I:\FreetokenModels\.convert-Ornith-1.5-35B-A3B-uncensored-NVFP4
[13:20:40] stdout/INFO    tensors: 463 weight + 240 experts_bank
[13:20:40] stdout/INFO    FTW: 21.49 GiB across 3 shard(s) (<= 8.0 GiB each)
[13:20:40] stdout/INFO    quant_format: nvfp4  fingerprint=6107af16a5c01426
[13:20:40] stdout/INFO    converted in 35.6s
[13:20:41] stdout/INFO  checkpoint Ornith-1.5-35B-A3B-uncensored-NVFP4 exited with code 0
[13:20:42] stderr/INFO  kernel warmup started for Ornith-1.5-35B-A3B-uncensored-NVFP4
[13:20:42] stderr/INFO  kernel warmup for Ornith-1.5-35B-A3B-uncensored-NVFP4 exited with exit code: 2: Use "ft --version" to print the FreeToken version.
[13:20:50] cmd/INFO  ft serve --model I:\FreetokenModels\Ornith-1.5-35B-A3B-uncensored-NVFP4 --port 1919 --moe-backend hybrid --max-running-requests 4 --memory-ratio 0.95 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420
[13:20:50] stdout/INFO  serve started (pid=10396 model=I:\FreetokenModels\Ornith-1.5-35B-A3B-uncensored-NVFP4 port=1919)
[13:20:53] stdout/INFO  [2026-08-29|13:20:53] INFO     Parsed arguments:
[13:20:53] stdout/INFO  ServerArgs(model_path='I:\\FreetokenModels\\Ornith-1.5-35B-A3B-uncensored-NVFP4', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, max_running_req=4, attention_backend='auto', moe_backend='hybrid', nvfp4_backend='triton', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=True, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=0, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.95, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=42000', _ipc_base_port=3340, server_host='127.0.0.1', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='Ornith-1.5-35B-A3B-uncensored-NVFP4', tool_call_parser='qwen3_coder', reasoning_parser='qwen3', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=(), gpu_assigned=None)
[13:20:53] stdout/INFO  [2026-08-29|13:20:53|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=20, top_p=0.95
[13:20:53] stdout/INFO  INFO:     Started server process [42000]
[13:20:53] stdout/INFO  INFO:     Waiting for application startup.
[13:20:53] stdout/INFO  INFO:     Application startup complete.
[13:20:53] stdout/INFO  INFO:     Uvicorn running on http://127.0.0.1:1919 (Press CTRL+C to quit)
[13:20:55] stdout/INFO  C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py:81: FutureWarning: torch.cuda._set_allocator_settings is deprecated. Use torch._C._accelerator_setAllocatorSettings instead.
[13:20:55] stdout/INFO    scheduler = Scheduler(args)
[13:20:55] stdout/INFO  [2026-08-29|13:20:55|core|rank=0] INFO     Enabled expandable_segments (override via PYTORCH_ALLOC_CONF)
[13:20:55] stdout/INFO  [2026-08-29|13:20:55|core|rank=0] INFO     Auto-selected attention backend: triton
[13:20:55] stdout/INFO  [2026-08-29|13:20:55|core|rank=0] INFO     Resolved config: moe_backend='hybrid', attention_backend='triton', cache_type='hybrid_radix', page_size=1
[13:20:55] stdout/INFO  [2026-08-29|13:20:55|core|rank=0] INFO     Free memory before loading model: 22.44 GiB
[13:20:55] stdout/INFO  C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\torch\utils\_device.py:116: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:39.)
[13:20:55] stdout/INFO    return func(*args, **kwargs)
[13:20:58] stdout/INFO  Loading weights (FTW):  70%|███████   | 3.20G/4.56G [00:01<00:00, 2.57GB/s]
[13:20:58] stdout/INFO  Process freetoken-TP0-scheduler:
[13:20:58] stdout/ERROR  [2026-08-29|13:20:58|FrontendAPI] ERROR    Backend supervisor: KeyError: 'model.layers.0.mlp.shared_expert.gate_up_proj.weight_scale'
[13:20:58] stdout/INFO  Traceback (most recent call last):
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 314, in _bootstrap
[13:20:58] stdout/INFO      self.run()
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 108, in run
[13:20:58] stdout/INFO      self._target(*self._args, **self._kwargs)
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 81, in _run_scheduler
[13:20:58] stdout/INFO      scheduler = Scheduler(args)
[13:20:58] stdout/INFO                  ^^^^^^^^^^^^^^^
[13:20:58] stdout/INFO    File "python/freetoken/scheduler/scheduler.py", line 65, in freetoken.scheduler.scheduler.Scheduler.__init__
[13:20:58] stdout/INFO    File "python/freetoken/engine/engine.py", line 363, in freetoken.engine.engine.Engine.__init__
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 48, in load_state_dict
[13:20:58] stdout/INFO      param.load_state_dict(
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 48, in load_state_dict
[13:20:58] stdout/INFO      param.load_state_dict(
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 96, in load_state_dict
[13:20:58] stdout/INFO      op.load_state_dict(state_dict, prefix=_concat_prefix(prefix, str(i)), _internal=True)
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 48, in load_state_dict
[13:20:58] stdout/INFO      param.load_state_dict(
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 48, in load_state_dict
[13:20:58] stdout/INFO      param.load_state_dict(
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\layers\base.py", line 48, in load_state_dict
[13:20:58] stdout/INFO      param.load_state_dict(
[13:20:58] stdout/INFO    File "C:\Users\Koolio\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\kernel\triton\nvfp4_linear.py", line 867, in load_state_dict
[13:20:58] stdout/INFO      s = state_dict.pop(_concat_prefix(prefix, "weight_scale"))
[13:20:58] stdout/INFO          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[13:20:58] stdout/INFO  KeyError: 'model.layers.0.mlp.shared_expert.gate_up_proj.weight_scale'
[13:21:06] health/ERROR  KeyError: 'model.layers.0.mlp.shared_expert.gate_up_proj.weight_scale'  (×5)
[13:21:08] stdout/ERROR  [2026-08-29|13:21:08|FrontendAPI] ERROR    Backend worker is gone and cannot be restarted; stopping the API server
[13:21:08] stdout/INFO  serve exited with code 15 (pid=10396)

[JIAQI13]

This is the same crash family as #164/#183: the shared expert is built as an NVFP4 merged linear (which expects shared_expert.gate_up_proj.weight_scale), but the checkpoint's shared expert is not NVFP4 — so the key is never produced during weight loading.

Two fixes are in review:

• fix(qwen3_5_moe): detect shared-expert quant for modelopt mixed checkpoints #294 — modelopt MIXED_PRECISION checkpoints (per-tensor FP8 shared expert, e.g. apodex/Apodex-1.1-mini-NVFP4): the shared-expert quant is now read from quantized_layers, and the dense pass only keeps native FP8 for attn/GDN.
  fix(qwen3_5_moe): detect shared-expert quant for modelopt mixed checkpoints #294

• feat(qwen3_5_moe): support compressed-tensors NVFP4 experts with block-fp8 dense side #296 — compressed-tensors / llm-compressor mixed checkpoints (128×128 block-FP8 shared expert + NVFP4 experts, e.g. kyaky/Qwen3.6-35B-A3B-Uncensored-NVFP4 and the abliterated exports in [Bug] ft checkpoint fails on Qwen3.6-35B-A3B abliterated NVFP4 checkpoints: Missing MoE expert source layers #263): adds a compressed-tensors NVFP4 expert bank loader and routes the dense side through the block-FP8 modules. A/B verified: main fails conversion/load, the PR converts (20.19 GiB FTW, 240 expert banks) and serves.
  feat(qwen3_5_moe): support compressed-tensors NVFP4 experts with block-fp8 dense side #296

If you can share the quantization_config block (or the Hugging Face id) of the Ornith-1.5-35B-A3B-uncensored-NVFP4 checkpoint — specifically quant_method and how *.mlp.shared_expert.* is quantized — it should map to one of the two. My guess from the uncensored-NVFP4 naming is the compressed-tensors family (covered by #296), but I have not verified that exact checkpoint.

[XeonG]

This is the same crash family as #164/#183: the shared expert is built as an NVFP4 merged linear (which expects shared_expert.gate_up_proj.weight_scale), but the checkpoint's shared expert is not NVFP4 — so the key is never produced during weight loading.

Two fixes are in review:

* **[fix(qwen3_5_moe): detect shared-expert quant for modelopt mixed checkpoints #294](https://github.com/FlashML-org/FreeToken/pull/294)** — modelopt `MIXED_PRECISION` checkpoints (per-tensor FP8 shared expert, e.g. `apodex/Apodex-1.1-mini-NVFP4`): the shared-expert quant is now read from `quantized_layers`, and the dense pass only keeps native FP8 for attn/GDN.
  [fix(qwen3_5_moe): detect shared-expert quant for modelopt mixed checkpoints #294](https://github.com/FlashML-org/FreeToken/pull/294)

* **[feat(qwen3_5_moe): support compressed-tensors NVFP4 experts with block-fp8 dense side #296](https://github.com/FlashML-org/FreeToken/pull/296)** — compressed-tensors / llm-compressor mixed checkpoints (128×128 block-FP8 shared expert + NVFP4 experts, e.g. `kyaky/Qwen3.6-35B-A3B-Uncensored-NVFP4` and the abliterated exports in [[Bug] ft checkpoint fails on Qwen3.6-35B-A3B abliterated NVFP4 checkpoints: Missing MoE expert source layers #263](https://github.com/FlashML-org/FreeToken/issues/263)): adds a compressed-tensors NVFP4 expert bank loader and routes the dense side through the block-FP8 modules. A/B verified: `main` fails conversion/load, the PR converts (20.19 GiB FTW, 240 expert banks) and serves.
  [feat(qwen3_5_moe): support compressed-tensors NVFP4 experts with block-fp8 dense side #296](https://github.com/FlashML-org/FreeToken/pull/296)

If you can share the quantization_config block (or the Hugging Face id) of the Ornith-1.5-35B-A3B-uncensored-NVFP4 checkpoint — specifically quant_method and how *.mlp.shared_expert.* is quantized — it should map to one of the two. My guess from the uncensored-NVFP4 naming is the compressed-tensors family (covered by #296), but I have not verified that exact checkpoint.

It was this model: https://huggingface.co/ressl/Ornith-1.5-35B-A3B-uncensored-NVFP4