Complete reference code and documentation for the comprehensive 6-part blog series on fine-tuning small language models using Docker Desktop.
This repository accompanies the detailed blog series:
- Part 1: Setup and Environment - Docker development environment with CUDA support
- Part 2: Data Preparation and Model Selection - High-quality dataset creation and model selection
- Part 3: Fine-Tuning with Unsloth - Efficient training with LoRA adapters
- Part 4: Evaluation and Testing - Comprehensive evaluation framework
- Part 5: Deployment with Ollama and Docker - Production deployment
- Part 6: Production, Monitoring, and Scaling - Enterprise operations
- Docker Desktop with GPU support
- Python 3.10+
- 16GB+ RAM (32GB+ recommended)
- NVIDIA GPU with 8GB+ VRAM (optional but recommended)
git clone https://github.com/saptak/fine-tuning-small-llms.git
cd fine-tuning-small-llms
# Set up environment
cp .env.example .env
# Edit .env with your configuration
# Start the development environment
docker-compose up -dfine-tuning-small-llms/
├── part1-setup/ # Development environment setup
│ ├── src/ # Setup scripts and utilities
│ ├── configs/ # Docker and environment configs
│ ├── scripts/ # Installation and setup scripts
│ └── docs/ # Part 1 documentation
├── part2-data-preparation/ # Dataset creation and validation
│ ├── src/ # Data processing utilities
│ ├── examples/ # Example datasets
│ └── scripts/ # Data preparation scripts
├── part3-training/ # Model fine-tuning
│ ├── src/ # Training scripts
│ ├── notebooks/ # Jupyter notebooks
│ └── configs/ # Training configurations
├── part4-evaluation/ # Model evaluation and testing
│ ├── src/ # Evaluation frameworks
│ ├── tests/ # Test suites
│ └── scripts/ # Evaluation scripts
├── part5-deployment/ # Production deployment
│ ├── src/ # API and web interfaces
│ ├── docker/ # Deployment containers
│ └── configs/ # Production configs
├── part6-production/ # Monitoring and optimization
│ ├── src/ # Production utilities
│ ├── monitoring/ # Grafana dashboards and configs
│ └── scripts/ # Production scripts
├── docker/ # Docker configurations
│ ├── images/ # Custom Docker images
│ └── compose/ # Docker Compose files
├── data/ # Training datasets
├── models/ # Model storage
└── docs/ # Additional documentation
- Environment Setup: Complete Docker-based development environment
- Data Engineering: High-quality dataset creation and validation techniques
- Model Training: Efficient fine-tuning with Unsloth and LoRA adapters
- Evaluation: Comprehensive testing frameworks and A/B testing
- Deployment: Production-ready APIs and web interfaces
- Operations: Monitoring, security, scaling, and cost optimization
- Unsloth - 80% faster, 80% less memory LLM fine-tuning
- Docker Desktop - Containerized development environment
- Ollama - Local LLM serving and inference
- FastAPI - High-performance API framework
- Streamlit - Interactive web interfaces
- Prometheus + Grafana - Monitoring and visualization
graph TB
A[Data Preparation] --> B[Model Training]
B --> C[Evaluation]
C --> D[Deployment]
D --> E[Production Monitoring]
subgraph "Part 1-2: Foundation"
A
F[Environment Setup]
end
subgraph "Part 3-4: Training & Testing"
B
C
end
subgraph "Part 5-6: Production"
D
E
end
Our approach achieves:
- 80% Memory Reduction with Unsloth optimization
- 2x Faster Training compared to standard fine-tuning
- Sub-second Inference for SQL generation tasks
- 99.9% Uptime with proper deployment configuration
- <$10/day operational costs for moderate usage
# Fine-tune a model for SQL generation
python part3-training/src/train_sql_model.py --dataset data/sql_dataset.json
# Deploy and test
curl -X POST "http://localhost:8000/generate-sql" \
-H "Content-Type: application/json" \
-d '{"instruction": "Find all users who registered last month"}'# Train for code documentation
python part3-training/src/train_code_docs.py --dataset data/code_docs_dataset.json# Train for customer support responses
python part3-training/src/train_support_model.py --dataset data/support_dataset.json- JWT Authentication for API access
- Rate Limiting and request throttling
- Input Validation and sanitization
- HTTPS/TLS encryption
- Web Application Firewall (WAF)
- Secrets Management with environment variables
- Real-time Metrics with Prometheus
- Custom Dashboards with Grafana
- Distributed Tracing for request flows
- Cost Tracking and optimization
- Automated Alerting for issues
- Setup Environment (Part 1)
cd part1-setup && ./scripts/setup_environment.sh- Prepare Data (Part 2)
cd part2-data-preparation && python src/create_dataset.py- Train Model (Part 3)
cd part3-training && python src/fine_tune_model.py- Evaluate Results (Part 4)
cd part4-evaluation && python src/run_evaluation.py- Deploy to Production (Part 5)
cd part5-deployment && ./scripts/deploy.sh- Monitor and Scale (Part 6)
cd part6-production && ./scripts/setup_monitoring.sh- CPU: 8+ cores recommended
- RAM: 16GB minimum, 32GB+ recommended
- GPU: NVIDIA GPU with 8GB+ VRAM (optional but recommended)
- Storage: 100GB+ free space
- OS: Linux, macOS, or Windows with WSL2
- Docker: Latest version with GPU support
- Python: 3.10 or higher
- CUDA: 11.8+ (if using GPU)
We welcome contributions! Please see our Contributing Guidelines for details.
# Clone the repository
git clone https://github.com/saptak/fine-tuning-small-llms.git
cd fine-tuning-small-llms
# Install development dependencies
pip install -r requirements-dev.txt
# Run tests
pytest tests/
# Format code
black src/
isort src/This project is licensed under the MIT License - see the LICENSE file for details.
- Unsloth Team for the amazing optimization framework
- Hugging Face for the transformers library
- Ollama Team for local LLM serving
- Docker Team for containerization platform
- Documentation: Check the
docs/directory for detailed guides - Issues: Report bugs and request features in GitHub Issues
- Discussions: Join the conversation in GitHub Discussions
- Unsloth - Fast LLM fine-tuning
- Ollama - Local LLM serving
- LangChain - LLM application framework
- Transformers - State-of-the-art ML models
⭐ Star this repository if you find it helpful!
Happy Fine-Tuning! 🚀