Fill this in with your own measured numbers from M0–M6. The point isn't the numbers themselves - it's that you can explain them with the roofline. This is the artifact you put on your CV / share.
Author: your name Date: YYYY-MM-DD Repo/notebook links: ...
| CPU (M0-M4) | GPU (M5-M6) | |
|---|---|---|
| Model | e.g. Xeon Gold 5220R (Cascade Lake) | e.g. NVIDIA T4 (Colab) |
| Cores / SMs | 12 physical | 40 SMs |
| ISA / Features | AVX2, AVX-512, bf16? | CUDA cap, tensor cores |
| RAM / VRAM | 39 GB | 15 GB |
| How obtained | native / droplet / Docker | Colab T4 |
(paste the sysinfo / cell-1 output for each)
| Anchor | CPU | GPU |
|---|---|---|
| Peak compute fp32 (GFLOP/s) | ||
| Peak compute bf16 (GFLOP/s) | tf32/fp16/bf16 → | |
| Peak memory bandwidth (GB/s) | ||
| Ridge point (FLOP/byte) | per precision |
One-line takeaway: e.g. "GPU compute is ~30× the CPU, but memory bandwidth only ~20×, so memory-bound ops close the gap."
| Op | CPU GFLOP/s | GPU GFLOP/s | Bound by | Why |
|---|---|---|---|---|
| matmul_1024_4096 | compute | high arithmetic intensity | ||
| conv2d 3×3 | compute | |||
| attention (SDPA) | compute | |||
| maxpool / upsample | memory | AI ≈ 0.2 FLOP/byte |
| Variant | GFLOP/s | Speedup vs naive | What changed |
|---|---|---|---|
| naive | 1× | ||
| tiled (cache blocking) | reuse data in cache | ||
| tiled + OpenMP | all cores | ||
| oneDNN benchmark (M1.5) | + SIMD + expert tuning (library ceiling) |
Gap analysis: why is your hand kernel still Nx below the library? (SIMD width, blocking, packing, …)
ISA note (M1.5): AVX-512 vs ONEDNN_MAX_CPU_ISA=avx2 – which was faster, and did you see throttling?
| Precision | CPU GFLOP/s | GPU GFLOP/s | Accuracy cost |
|---|---|---|---|
| fp32 | reference | ||
| bf16 / fp16 | relative error | ||
| int8 | -- | relative error |
Takeaway: when did low precision help, when did it hurt, and why (hardware support)?
- Fused op: relu(x+y) / SiLU / … Correct vs PyTorch: yes/no.
- eager: … ms – Triton: … ms (…×) –
torch.compile: … ms (…×). - Insight: why the fused kernel wins (memory passes), and whether hand-tuning beat
torch.compile.
| CPU (llama.cpp) | GPU (transformers) | |
|---|---|---|
| Model | Qwen2.5 | Qwen2.5-0.5B |
| tokens/s @ batch 1 | ||
| tokens/s @ best batch | (thread sweep) | (batch sweep) |
| Quantization used | Q4_K | fp16 / 8-bit |
Throughput vs latency: where did batching/threads stop helping, and why?
- 3–5 bullets: the biggest surprises, the clearest roofline lesson, and one thing you'd optimize next.
- Optional: a roofline plot (compute ceiling, BW ceiling, your ops as points) for CPU and GPU.
Method note: GPU numbers on a shared Colab T4 are indicative (throttled/shared). For publishable figures, re-run on a dedicated GPU and report variance across 3+ runs.