Skip to content

BENCH-005: Ternary vs Binary — Extended Multi-Dataset Validation #494

@gHashTag

Description

@gHashTag

Objective

Validate H1 on two datasets (MNIST + CIFAR-10) across 5 number formats.

Datasets

Dataset Images Classes Resolution Channels Complexity
MNIST 70k 10 28×28 1 (gray) Low
CIFAR-10 60k 10 32×32 3 (RGB) Medium

Formats Under Test

Format Bits Bytes/weight Compression vs FP32
FP32 32 4.0 1× (baseline)
GF16 16 2.0
FP16 16 2.0
BF16 16 2.0
Ternary 2 0.125 32×

Model Architecture

  • MLP: 784→128→10 (MNIST) / 3072→256→128→10 (CIFAR-10)

Success Criteria

  • GF16 gap vs FP32: ≤0.5% on both datasets
  • Ternary gap vs FP32: ≤2% on MNIST, ≤5% on CIFAR-10

Implementation Steps

  1. Create src/cifar10_loader.zig — parse CIFAR-10 binary batches
  2. Extend src/bench_format_comparison.zig — multi-dataset benchmark
  3. Run 5 formats × 2 datasets × 3 seeds = 30 runs
  4. Generate results table + CSV export
  5. Update docs/proposals/BENCH_005_RESULTS.md

φ² + 1/φ² = 3 | TRINITY

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions