feat: fit the 18B model on a 24 GB GPU + 64 GB RAM box (staged T5, eager FP8, FP8-only deepcopy) - #20

Open

yangohuang wants to merge 3 commits into

Soul-AILab:mainfrom

yangohuang:feat/consumer-gpu-memory

Commits on Jul 22, 2026

feat: pre-encode prompts and release T5 before loading the DiT
yangohuang
committed
7c56d1b
fix: support FP8-only source in FP8Linear.__deepcopy__
yangohuang
committed
2b96986
feat: eager per-block FP8 materialization under --fp8_gemm --block_offload
yangohuang
committed
e6d0451