Skip to content

Conversation

yansigit
Copy link

Checklist:

This PR adds MLX support for Apple Silicone devices. #4833

@Copilot Copilot AI review requested due to automatic review settings August 9, 2025 05:00
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds MLX support for Apple Silicon devices, enabling text generation using Apple's MLX framework. MLX is optimized for Apple Silicon hardware and provides an alternative to traditional GPU-based inference.

  • Adds MLX as a new loader option for Apple Silicon devices
  • Creates a comprehensive MLX model implementation with streaming and non-streaming generation
  • Integrates MLX model detection and parameter mapping for the webui

Reviewed Changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
requirements/portable/requirements_apple_silicon.txt Adds mlx-lm dependency for portable installations
requirements/full/requirements_apple_silicon.txt Adds mlx-lm dependency for full installations
modules/text_generation.py Integrates MLXModel into generation pipeline
modules/models_settings.py Adds MLX model detection logic for Apple Silicon
modules/models.py Adds MLX loader function and unload support
modules/mlx_loader.py Implements complete MLX model class with generation capabilities
modules/loaders.py Adds MLX loader configuration and sampling parameters

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant