Project Idea ?

[thinley4]

Suggest any project to build

[Cfomodz]

Beginner Projects

1. Tokenizer Comparison Tool

Compare BPE, WordPiece, and whitespace tokenizers with visual token splits

• Chapter: 2
• Bonus: ch02/05_bpe-from-scratch

2. Attention Visualizer

Interactive tool showing attention scores and multi-head patterns

• Chapter: 3
• Bonus: ch04/04_gqa, ch04/06_swa

3. Mini Text Generator

Small GPT trained on focused dataset (Shakespeare, recipes, code)

• Chapters: 4-5
• Bonus: ch04/03_kv-cache

Intermediate Projects

4. Domain-Specific LLM

Pretrain on specialized corpus (medical, legal, programming)

• Chapter: 5
• Bonus: ch05/03_bonus_pretraining_on_gutenberg

5. Multi-Task Classifier

Single model for sentiment, spam detection, and topic classification

• Chapter: 6
• Bonus: ch06/02_bonus_additional-experiments

6. Code Assistant

Finetune for code explanation, generation, and documentation

• Chapters: 5-7

Advanced Projects

7. Instruction Chatbot with DPO

Conversational AI with preference learning and alignment

• Chapter: 7
• Bonus: ch07/04_preference-tuning-with-dpo

8. Mixture-of-Experts Model

Sparse model with expert routing and load balancing

• Bonus: ch04/07_moe, ch05/11_qwen3

9. Production LLM Server

Optimized serving with quantization, KV-cache, and REST API

• Multiple chapters
• Bonus: ch05/08_memory_efficient_weight_loading

10. RAG Question Answering

Document retrieval with semantic search and source attribution

• Chapters: 2-7

Quick Start

1. Pick project matching your chapter progress
2. Use repository code as foundation
3. Start with small datasets/models
4. Leverage bonus materials extensively