-
Notifications
You must be signed in to change notification settings - Fork 5.8k
Add MLX support #7140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add MLX support #7140
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds MLX support for Apple Silicon devices, enabling text generation using Apple's MLX framework. MLX is optimized for Apple Silicon hardware and provides an alternative to traditional GPU-based inference.
- Adds MLX as a new loader option for Apple Silicon devices
- Creates a comprehensive MLX model implementation with streaming and non-streaming generation
- Integrates MLX model detection and parameter mapping for the webui
Reviewed Changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
File | Description |
---|---|
requirements/portable/requirements_apple_silicon.txt | Adds mlx-lm dependency for portable installations |
requirements/full/requirements_apple_silicon.txt | Adds mlx-lm dependency for full installations |
modules/text_generation.py | Integrates MLXModel into generation pipeline |
modules/models_settings.py | Adds MLX model detection logic for Apple Silicon |
modules/models.py | Adds MLX loader function and unload support |
modules/mlx_loader.py | Implements complete MLX model class with generation capabilities |
modules/loaders.py | Adds MLX loader configuration and sampling parameters |
Checklist:
This PR adds MLX support for Apple Silicone devices. #4833