Support for 1bit models

[leoxs22]

I just downloaded https://huggingface.co/prism-ml/Bonsai-8B-mlx-1bit and got

Error when loading model: ValueError: [quantize] The requested number of bits 1 is not supported. The supported bits are 2, 3, 4, 5, 6 and 8.

I'm on macos, arm, lm studio version 0.4.8

[alextuan1024]

Same.
The error should be emitted from mlx_lm.

mlx_lm.server --model "prism-ml/Bonsai-8B-mlx-1bit"

[reski-rukmantiyo]

hi all,

tried to load the model using above command

mlx_lm.server --model "prism-ml/Bonsai-8B-mlx-1bit"

but still failed. same error message.

ValueError: [quantize] The requested number of bits 1 is not supported. The supported bits are 2, 3, 4, 5, 6 and 8.
Download complete: : 0.00B [00:00, ?B/s]

Following are the logs:

((venv) ) reski@Reskis-Mac-mini mlx-engine % mlx_lm.server --model "prism-ml/Bonsai-8B-mlx-1bit"
2026-04-30 03:52:51,227 - INFO - HTTP Request: GET https://huggingface.co/api/models/prism-ml/Bonsai-8B-mlx-1bit/revision/main "HTTP/1.1 200 OK"
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 23172.95it/s]
Download complete: : 0.00B [00:00, ?B/s]              Traceback (most recent call last):                                                                                                                             | 0/12 [00:00<?, ?it/s]
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/bin/mlx_lm.server", line 6, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx_lm/server.py", line 1885, in main
    run(args.host, args.port, ModelProvider(args))
                              ^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx_lm/server.py", line 308, in __init__
    self.load(self.cli_args.model, draft_model_path="default_model")
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx_lm/server.py", line 342, in load
    model, tokenizer = load(
                       ^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx_lm/utils.py", line 491, in load
    model, config = load_model(model_path, lazy, model_config=model_config)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx_lm/utils.py", line 366, in load_model
    _quantize(quantization)
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx_lm/utils.py", line 357, in _quantize
    nn.quantize(
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx/nn/layers/quantized.py", line 94, in quantize
    leaves = tree_map_with_path(_maybe_quantize, leaves, is_leaf=Module.is_module)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx/utils.py", line 108, in tree_map_with_path
    k: tree_map_with_path(
       ^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx/utils.py", line 108, in tree_map_with_path
    k: tree_map_with_path(
       ^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx/utils.py", line 95, in tree_map_with_path
    return fn(path, tree, *rest)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx/nn/layers/quantized.py", line 76, in _maybe_quantize
    return m.to_quantized(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx/nn/layers/embedding.py", line 53, in to_quantized
    return QuantizedEmbedding.from_embedding(self, group_size, bits, mode)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx/nn/layers/quantized.py", line 189, in from_embedding
    ql = cls(embedding_dims, dims, group_size, bits, mode=mode)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Apps/home/reski/Github/mlx-engine/venv/lib/python3.12/site-packages/mlx/nn/layers/quantized.py", line 134, in __init__
    self.weight, self.scales, *biases = mx.quantize(
                                        ^^^^^^^^^^^^
ValueError: [quantize] The requested number of bits 1 is not supported. The supported bits are 2, 3, 4, 5, 6 and 8.
Download complete: : 0.00B [00:00, ?B/s]

[lustfeind]

Bump! Add support for prism-ml: Ternary Boinsai 27b

[DReffects]

second this

[RichardoC]

mlx still doesn't support it, but gguf 1bit quants seem to be working well