-
-
Notifications
You must be signed in to change notification settings - Fork 1
BENCH-005: Ternary vs Binary — Extended Multi-Dataset Validation #494
Copy link
Copy link
Open
Description
Objective
Validate H1 on two datasets (MNIST + CIFAR-10) across 5 number formats.
Datasets
| Dataset | Images | Classes | Resolution | Channels | Complexity |
|---|---|---|---|---|---|
| MNIST | 70k | 10 | 28×28 | 1 (gray) | Low |
| CIFAR-10 | 60k | 10 | 32×32 | 3 (RGB) | Medium |
Formats Under Test
| Format | Bits | Bytes/weight | Compression vs FP32 |
|---|---|---|---|
| FP32 | 32 | 4.0 | 1× (baseline) |
| GF16 | 16 | 2.0 | 2× |
| FP16 | 16 | 2.0 | 2× |
| BF16 | 16 | 2.0 | 2× |
| Ternary | 2 | 0.125 | 32× |
Model Architecture
- MLP: 784→128→10 (MNIST) / 3072→256→128→10 (CIFAR-10)
Success Criteria
- GF16 gap vs FP32: ≤0.5% on both datasets
- Ternary gap vs FP32: ≤2% on MNIST, ≤5% on CIFAR-10
Implementation Steps
- Create
src/cifar10_loader.zig— parse CIFAR-10 binary batches - Extend
src/bench_format_comparison.zig— multi-dataset benchmark - Run 5 formats × 2 datasets × 3 seeds = 30 runs
- Generate results table + CSV export
- Update
docs/proposals/BENCH_005_RESULTS.md
φ² + 1/φ² = 3 | TRINITY
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels