Skip to content

perf(train): use U8 confusion codes when supported - #6224

Open
Mikyx-1 wants to merge 2 commits into
tracel-ai:mainfrom
Mikyx-1:perf/compact-confusion-codes
Open

Mikyx-1 wants to merge 2 commits into
tracel-ai:mainfrom
Mikyx-1:perf/compact-confusion-codes

Conversation

@Mikyx-1

@Mikyx-1 Mikyx-1 commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Checklist

  • I have reviewed and understand the changes in this PR.
  • Confirmed that cargo run-checks has been executed.
  • Made sure the book is up to date with changes in this PR.
  • Added or updated changelog entries for user-visible changes, if applicable, following the
    changelog policy in CONTRIBUTING.md.

Related Issues/PRs

Closes #6223.

Changes

Use U8 confusion codes when supported, with the original integer expression as the fallback.
Keep count reductions and metric behavior unchanged. Add regression tests and a changelog entry.

The four codes need only two bits. U8 works with existing tensor operations; packing four codes
per byte would require separate encoding/counting logic and performance measurements.

Benchmarks

Complete Precision updates, threshold 0.5, micro reduction. Flex CPU, Intel i5-12400F,
release mode, Rust 1.98.0, default I32, one worker. Medians of nine paired rounds
(fifteen for 262,144 classes):

Batch × classes Previous µs U8 µs Speedup
32 × 10 5.29 5.05 1.05×
32 × 1,000 37.70 21.36 1.77×
256 × 4,096 1,441.43 766.78 1.88×
1,024 × 4,096 18,825.03 7,589.64 2.48×
1 × 262,144 333.44 170.97 1.95×

At 256 × 4,096, additional allocation peak falls from 9 MiB to 6 MiB. At 1 × 262,144,
macro gains are only 1.02–1.11× across the three metrics; top-k performance is largely unchanged.

Testing

  • cargo test -p burn-train --lib metric:: --no-default-features --locked: 281 passed.
  • cargo run-checks: passed with Flex; existing ignored tests remained skipped.
  • Paired release benchmarks: 822 configurations passed exact-output checks. Allocation counting
    ran separately from latency measurements.
  • GPU execution and the unsupported-U8 fallback branch were not tested.

AI Usage

Codex autonomously implemented and tested the change, ran validation and benchmarks, and updated
the changelog following user instructions. It drafted the issue and PR descriptions and created
the issue and this PR.

@codecov

codecov Bot commented Oct 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.15%. Comparing base (dd31726) to head (54274d8).
⚠️ Report is 1 commits behind head on main.

❌ Your project check has failed because the head coverage (66.15%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6224      +/-   ##
==========================================
- Coverage   66.16%   66.15%   -0.02%     
==========================================
  Files         878      878              
  Lines      127748   127748              
==========================================
- Hits        84528    84513      -15     
- Misses      43220    43235      +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(train): reduce confusion-code storage for classification metrics

1 participant