feat: fit the 18B model on a 24 GB GPU + 64 GB RAM box (staged T5, eager FP8, FP8-only deepcopy) - #20

Open

yangohuang wants to merge 3 commits into

Enhance your code review process with GitHub Actions

GitHub Actions make it easy to automate all your software workflows, now with world-class CI/CD.
Build, test, and deploy your code right from GitHub. Learn more about GitHub Actions.

Linux, macOS, Windows, and containers

Navigation Menu

Uh oh!

feat: fit the 18B model on a 24 GB GPU + 64 GB RAM box (staged T5, eager FP8, FP8-only deepcopy) - #20

feat: fit the 18B model on a 24 GB GPU + 64 GB RAM box (staged T5, eager FP8, FP8-only deepcopy)#20
yangohuang wants to merge 3 commits into
Soul-AILab:mainfrom
yangohuang:feat/consumer-gpu-memory

Enhance your code review process with GitHub Actions

Linux, macOS, Windows, and containers

Matrix builds

Any language

Live logs

Built-in secret store

Multi-container testing