Add Local AI Studio: Complete local AI development environment#150
Draft
chuckeelord wants to merge 2 commits into0x4m4:masterfrom
Draft
Add Local AI Studio: Complete local AI development environment#150chuckeelord wants to merge 2 commits into0x4m4:masterfrom
chuckeelord wants to merge 2 commits into0x4m4:masterfrom
Conversation
Complete modular Python package providing: - Model management: GGUF model discovery, HuggingFace downloads, Ollama integration, quantization utilities (Q4_K_M/Q5_K_M), LoRA/QLoRA adapter management with training config generation - Chat engine: multi-turn conversations with streaming, context window management, conversation persistence with tagging/categorization, export to JSON/Markdown - Tool framework: sandboxed Python execution, filesystem operations with root restrictions, web/API fetching, SQLite/CSV database queries, git operations with safety controls, pip package management - MCP server: full Model Context Protocol support exposing chat, tools, model management, and hardware info as MCP resources/tools/prompts - Gradio web GUI: 6-tab interface (Chat, Models, Settings, Tools, Hardware Monitor, History) with dark theme, inference presets (code/research/creative/roleplay), system prompt templates, real-time VRAM monitoring, profile import/export - CLI: subcommands for gui, mcp, chat, scan, info, config with full argument parsing and slash-commands in interactive mode - Hardware detection: NVIDIA GPU via nvidia-smi, CPU/RAM monitoring, automatic recommendations for quantization, context length, batch size, and thread count based on available VRAM Optimized for RTX 5060 (8GB VRAM) + Snapdragon X Plus hardware. All components pip-installable with optional dependency groups. https://claude.ai/code/session_01E8sDdaXo9Mry8phPH8KYgY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces Local AI Studio, a comprehensive local AI development environment with a web GUI, model management, chat interface, tool integration, and MCP server support. The system is optimized for consumer hardware (NVIDIA RTX 5060 + Snapdragon X Plus) and provides a fully self-contained AI development platform.
Key Changes
Core Architecture
config.py): Centralized configuration management with inference presets, system prompt templates, and directory layouthardware.py): GPU/CPU detection, VRAM/RAM monitoring, and hardware-aware recommendationsmodels/manager.py): Model registry, discovery, loading (GGUF via llama-cpp-python, Ollama integration), and lifecycle managementchat/engine.py): Multi-turn conversation orchestration with streaming, system prompts, and tool invocationchat/history.py): Save/load conversations with metadata, tagging, and search capabilitiesUser Interface
gui/app.py): Full-featured browser interface with:Tool Integration Framework
tools/executor.py): Unified tool registry and dispatcher with permission checkstools/python_sandbox.py): Sandboxed code execution with timeout and memory limitstools/filesystem.py): Restricted file operations within allowed directoriestools/web.py): URL fetching and web scraping with URL filteringtools/database.py): SQLite query execution with safety controlstools/git_tools.py): Version control operations with destructive action preventionModel Optimization
models/quantization.py): GGUF quantization format metadata and VRAM estimationmodels/lora.py): LoRA adapter discovery, indexing, and applicationProtocol Support
mcp/server.py): Model Context Protocol implementation for IDE/tool integration, exposing models, conversations, and tools as MCP resourcesCLI & Entry Point
__main__.py): Multiple entry points:gui: Launch web interfacemcp: Run MCP serverchat: Interactive CLI chatscan: Model discoveryinfo: Hardware informationconfig: Configuration displayNotable Implementation Details
Dependencies
psutil,requestsllama-cpp-python(with optional CUDA support)gradiofastmcpollama,transformers,peft(for LoRA)https://claude.ai/code/session_01E8sDdaXo9Mry8phPH8KYgY