With no index and no safetensors present, this returns an empty list, so the `shards` dict in
`ple_disk.py:63-87` stays empty and `ple_disk.py:88-89` raises:
```python
if sorted(shards) != list(range(len(shards))) or not shards:
raise ValueError(f"PLE shard indices are not contiguous 0..N-1: {sorted(shards)[:8]}")
Note the message is misleading for this failure mode: [] means "found no PLE shards in this
folder at all", but it reads as "found them and they are out of order", which points an operator at
checkpoint corruption rather than at a missing file.
Restoring only some of the 10 files produces the genuinely useful form, but the preview is still
confusing, because the shards are scattered across the files in lexicographic key order rather than
numeric order - which is what weight.py:229-231 documents ("scattered over the model-plefp8-*
shards in header (lexicographic) order"). Verified from the file headers:
model-plefp8-00000.safetensors -> shards [0, 1, 10, 100..109]
model-plefp8-00001.safetensors -> shards [11, 12, 110..120]
model-plefp8-00002.safetensors -> shards [13..18, 121..127]
model-plefp8-00003.safetensors -> shards [2, 3, 19..29]
...
model-plefp8-00009.safetensors -> shards [9, 90..99] + weight_scale
So with files 00005-00008 still outstanding the check reports
[0, 1, 2, 3, 4, 9, 10, 11], and with 00007-00008 outstanding it reports
[0, 1, 2, 3, 4, 5, 6, 9]. Both are simply the first eight indices present, not a
contiguous-prefix failure - which is worth saying in the message too.
Evidence
Converted output directory as ft checkpoint left it, before restoring anything (73 GiB, 223
files; it is 121 GiB now that the 10 PLE shards are back):
freetoken-00000.ftw .. freetoken-00009.ftw, 72.70 GiB total
freetoken_weight.json: 794 weight + 288 experts_bank tensors, and zero keys matching
ngram_embedding
- 0 files matching
*.safetensors; no model.safetensors.index.json
- all non-weight metadata was copied through, including the 192
layer-*-experts-*.complete.json
sidecars (they end in .json, so _WEIGHT_SUFFIXES does not skip them)
That last row is the tell: the copy step is working exactly as written. Only the weight-class-3
shards are missing, because only they are both .safetensors and not represented in the FTW.
What the 10 dropped shards contain, verified against the published model.safetensors.index.json
Independent of which is chosen, the ple_disk.py:88-89 message should distinguish "no PLE shards
found under <folder>" from "shards found but non-contiguous". The former is what a dropped-file
bug looks like, and the current wording sent me looking for checkpoint corruption first.
Workaround
Keep the 10 model-plefp8-*.safetensors in the directory that --model points at. The index is
not required: with only the PLE shards present, the iter_weight_files() fallback resolves them
correctly and does not pick up the .ftw files. I confirmed that directly - _ple_table_files()
on a directory holding 3 of the 10 shards and no index returned exactly those 3 paths, with
includes any .ftw? False.
With all 10 restored (each byte-exact against the published size, 47.68 GiB total) the table
resolves:
from freetoken.models.qwen4_exp.ple_disk import resolve_row_source
s = resolve_row_source(r"E:\LLM\models\RadixArk--Qwen3.8-Flash-Next-NVFP4")
len(s.paths), len(s.extent_base), s.total_rows, s.row_bytes, s.scale
(10, 128, 320001536, 160, 0.00019931793212890625)
and ft serve on the FTW directory now starts and generates, on the default --ple-backend disk:
$ curl -s http://127.0.0.1:1919/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"RadixArk--Qwen3.8-Flash-Next-NVFP4","messages":[{"role":"user","content":"The capital of France is"}],"max_tokens":48,"temperature":0.0}'
{"id":"chatcmpl-3","object":"chat.completion","created":1788456738,"model":"RadixArk--Qwen3.8-Flash-Next-NVFP4","choices":[{"index":0,"message":{"role":"assistant","content":"Paris","reasoning_content":"We need to respond to user: \"The capital of France is\". Likely wants completion. Need final concise. Ensure no hidden issues. Answer: Paris. Could complete sentence."},"finish_reason":"stop"}],"usage":{"prompt_tokens":57,"completion_tokens":40,"total_tokens":97}}
GET /health returns 200 and GET /v1/models reports context_length: 262144. That request took
14.5 s wall for 40 completion tokens (cold); one earlier request issued during engine warm-up
returned a JSON error body with no choices. The per-table scalar weight_scale is read from
model-plefp8-00009.safetensors and applied at lookup (ple_disk.py:261-262), so this path is
not the silent wrong-embedding case the checkpoint's qualification notes warn about.
So the only thing standing between a successful ft checkpoint and a servable directory is those
10 files, which the converter had already read once and then declined to carry over.
On a 128 GiB host, stay on the default --ple-backend disk. The pinned path puts the full
47.7 GiB table in host RAM on top of the offload expert banks; #293 reports 113 GiB worker RSS at
ready for that configuration, which does not fit here.
Disclosure
Drafted with AI assistance. Every file and line reference, byte count, tensor name, dtype, shape,
shard-to-file mapping, version string, log line and HTTP response above was read directly from the
installed 0.1.2+g0ee42b72c sources, from main at 03c28d2b via the GitHub API, from the
checkpoint's published model.safetensors.index.json and the restored shard headers, or from the
running server. The resolve_row_source tuple and the /v1/chat/completions response in the
workaround section are measured, not projected: the 10 shards are complete and byte-exact, and the
request was served by this host.
Note the message is misleading for this failure mode:
[]means "found no PLE shards in thisfolder at all", but it reads as "found them and they are out of order", which points an operator at
checkpoint corruption rather than at a missing file.
Restoring only some of the 10 files produces the genuinely useful form, but the preview is still
confusing, because the shards are scattered across the files in lexicographic key order rather than
numeric order - which is what
weight.py:229-231documents ("scattered over themodel-plefp8-*shards in header (lexicographic) order"). Verified from the file headers:
So with files 00005-00008 still outstanding the check reports
[0, 1, 2, 3, 4, 9, 10, 11], and with 00007-00008 outstanding it reports[0, 1, 2, 3, 4, 5, 6, 9]. Both are simply the first eight indices present, not acontiguous-prefix failure - which is worth saying in the message too.
Evidence
Converted output directory as
ft checkpointleft it, before restoring anything (73 GiB, 223files; it is 121 GiB now that the 10 PLE shards are back):
freetoken-00000.ftw..freetoken-00009.ftw, 72.70 GiB totalfreetoken_weight.json: 794weight+ 288experts_banktensors, and zero keys matchingngram_embedding*.safetensors; nomodel.safetensors.index.jsonlayer-*-experts-*.complete.jsonsidecars (they end in
.json, so_WEIGHT_SUFFIXESdoes not skip them)That last row is the tell: the copy step is working exactly as written. Only the weight-class-3
shards are missing, because only they are both
.safetensorsand not represented in the FTW.What the 10 dropped shards contain, verified against the published
model.safetensors.index.jsonIndependent of which is chosen, the
ple_disk.py:88-89message should distinguish "no PLE shardsfound under
<folder>" from "shards found but non-contiguous". The former is what a dropped-filebug looks like, and the current wording sent me looking for checkpoint corruption first.
Workaround
Keep the 10
model-plefp8-*.safetensorsin the directory that--modelpoints at. The index isnot required: with only the PLE shards present, the
iter_weight_files()fallback resolves themcorrectly and does not pick up the
.ftwfiles. I confirmed that directly -_ple_table_files()on a directory holding 3 of the 10 shards and no index returned exactly those 3 paths, with
includes any .ftw? False.With all 10 restored (each byte-exact against the published size, 47.68 GiB total) the table
resolves:
and
ft serveon the FTW directory now starts and generates, on the default--ple-backend disk:$ curl -s http://127.0.0.1:1919/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"RadixArk--Qwen3.8-Flash-Next-NVFP4","messages":[{"role":"user","content":"The capital of France is"}],"max_tokens":48,"temperature":0.0}'{"id":"chatcmpl-3","object":"chat.completion","created":1788456738,"model":"RadixArk--Qwen3.8-Flash-Next-NVFP4","choices":[{"index":0,"message":{"role":"assistant","content":"Paris","reasoning_content":"We need to respond to user: \"The capital of France is\". Likely wants completion. Need final concise. Ensure no hidden issues. Answer: Paris. Could complete sentence."},"finish_reason":"stop"}],"usage":{"prompt_tokens":57,"completion_tokens":40,"total_tokens":97}}GET /healthreturns 200 andGET /v1/modelsreportscontext_length: 262144. That request took14.5 s wall for 40 completion tokens (cold); one earlier request issued during engine warm-up
returned a JSON error body with no
choices. The per-table scalarweight_scaleis read frommodel-plefp8-00009.safetensorsand applied at lookup (ple_disk.py:261-262), so this path isnot the silent wrong-embedding case the checkpoint's qualification notes warn about.
So the only thing standing between a successful
ft checkpointand a servable directory is those10 files, which the converter had already read once and then declined to carry over.
On a 128 GiB host, stay on the default
--ple-backend disk. The pinned path puts the full47.7 GiB table in host RAM on top of the offload expert banks; #293 reports 113 GiB worker RSS at
ready for that configuration, which does not fit here.
Disclosure
Drafted with AI assistance. Every file and line reference, byte count, tensor name, dtype, shape,
shard-to-file mapping, version string, log line and HTTP response above was read directly from the
installed
0.1.2+g0ee42b72csources, frommainat03c28d2bvia the GitHub API, from thecheckpoint's published
model.safetensors.index.jsonand the restored shard headers, or from therunning server. The
resolve_row_sourcetuple and the/v1/chat/completionsresponse in theworkaround section are measured, not projected: the 10 shards are complete and byte-exact, and the
request was served by this host.