Add mlx_lm.server example

[losuler]

This one took me longer than I liked to figure out, so I wanted to add an example for anyone else who is coming across the same problem. Which was that without useModelName being the file path, mlx_lm.server is using the model name, which causes it to download the model to the huggingface cache path, instead of using the one I manually downloaded to the specified --model path.

Summary by CodeRabbit

• New Features
  • Adds a new model option: “Qwen3-Next-80B-A3B-Thinking-4bit.”
  • Includes a ready-to-use launch command (with port configuration) and an explicit model-name override for easier setup and reference.
  • Expands the public model registry without affecting existing models or startup/shutdown behavior.
• Chores
  • Minor formatting cleanup (added end-of-file newline).

[coderabbitai]

Walkthrough

Adds a new model entry to config.example.yaml under models with a multiline cmd to start mlx_lm.server and a useModelName override. No other files or runtime logic changed. Adds end-of-file newline.

Changes

Cohort / File(s)	Summary
Config: models registry config.example.yaml	Added model mlx-community/Qwen3-Next-80B-A3B-Thinking-4bit with a multiline cmd that runs mlx_lm.server --port ${PORT} and points --model /models/mlx-community/Qwen3-Next-80B-A3B-Thinking-4bit; added useModelName override; added trailing newline.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title "Add mlx_lm.server example" concisely and accurately summarizes the primary change in this PR—adding an mlx_lm.server example entry (in config.example.yaml) to demonstrate running a model with --model and useModelName. It is short, specific to the main change described in the PR objectives and raw_summary, and is easily understood by a teammate scanning history. The title contains no extraneous wording or ambiguity.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.

✨ Finishing touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment

---

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f79bee7 and 7a15540.

📒 Files selected for processing (1)

• config.example.yaml (2 hunks)

✅ Files skipped from review due to trivial changes (1)

• config.example.yaml

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

• Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
• Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Please see the documentation for more information.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c36986f and f79bee7.

📒 Files selected for processing (1)

• config.example.yaml (2 hunks)

🔇 Additional comments (1)

config.example.yaml (1)

256-256: LGTM

Preload entry unchanged functionally.

[olegshulyakov]

@losuler Does it support full set of llama.cpp endpoints?

[rosmur]

interesting.. didnt realize that llama-swap supports mlx_lm as well!

[losuler]

Does it support full set of llama.cpp endpoints?

I guess not, but it supports v1/completions¹, v1/chat/completions² and v1/models³, which I haven't noticed any problems using llama-swap with mlx-lm in this way on Open WebUI.

Footnotes

1. https://github.com/ml-explore/mlx-lm/blob/f318741784496dc2025dd7a4dea1ae698d21c610/tests/test_server.py#L73 ↩

2. https://github.com/ml-explore/mlx-lm/blob/f318741784496dc2025dd7a4dea1ae698d21c610/tests/test_server.py#L100 ↩

3. https://github.com/ml-explore/mlx-lm/blob/f318741784496dc2025dd7a4dea1ae698d21c610/tests/test_server.py#L173 ↩