XAI-driven augmentation & diagnostics for PyTorch vision: find model failures, fix with saliency-guided augmentation (ICD/AICD), prove with auditable reports.
Demo with audio (1280p). Original 4K recording: bnnr.dev
An HTML failure report for any PyTorch vision checkpoint — see where your model looks when it's wrong, then fix it with saliency-guided augmentation. MIT licensed. No retraining required for diagnostics.
Run bnnr analyze on a trained model and you get a self-contained HTML report: confusion matrix, top confused pairs, OptiCAM saliency overlays on the worst failures, and plain-language recommendations.
Try it without installing: sample analyze HTML report (MNIST, real run — confusion pairs, XAI heatmaps, recommendations).
- I have a trained checkpoint and validation looks fine — but I don't fully trust it. →
bnnr analyzegives an XAI failure report in under 5 minutes, no retraining. See where the model actually looks on its worst errors. - My augmentations are guesswork. → ICD/AICD condition augmentation on saliency maps, then a branch search keeps only the augmentations that measurably improve your validation metric.
- I need to prove model quality to stakeholders or compliance. → A portable
report.htmlplus a structured JSON audit artifact you can attach to a review.
BNNR also ships a training pipeline (bnnr train), a live dashboard, multi-label support, and object detection (YOLO / COCO-mini). Single-label classification is the focus for analyze. Supported tasks in v0.6.5: single-label classification, multi-label classification, object detection. Full docs: docs/README.md (analyze · detection · benchmarks).
If you already have a trained checkpoint, run diagnostics without retraining:
pip install bnnr
python3 -m bnnr analyze --model checkpoints/best.pt --data cifar10 --output ./analysis_out
open ./analysis_out/report.htmlYou get a self-contained report.html (metrics, confusion matrix, top confused pairs, OptiCAM saliency overlays on failures, recommendations) plus an analysis_report.json audit artifact.
No checkpoint yet? Open the sample HTML report from a real MNIST run — no install required.
See docs/analyze.md for the full workflow (custom datasets, torchvision models, multi-label).
pip install "bnnr[dashboard]"
# Zero flags — CIFAR-10 demo CNN, ICD preset, live dashboard (~1 min)
python3 -m bnnr demoInteractive wizard (prompts for dataset/preset; sample limits 128/64):
python3 -m bnnr quickstartFull CLI training with built-in defaults:
python3 -m bnnr train --dataset cifar10 --preset light --with-dashboardOpen http://127.0.0.1:8080/ for the live dashboard (QR code in terminal for mobile on the same Wi-Fi).
Advanced: pass --config path.yaml to override defaults.
BNNR uses saliency maps to guide augmentation, not random flips and crops. One Grad-CAM map yields two targeted augmentations: ICD hides the most salient region (so the model cannot lean on a shortcut), AICD hides the background (so it focuses on the object).
The masked tiles are filled with a configurable strategy (Gaussian blur, local mean, noise, or solid):
Figures from the Grad-CAM to ICD/AICD notebook. Method paper: ICD/AICD on Zenodo.
| Dataset | Without BNNR | BNNR branch search | RandAugment |
|---|---|---|---|
| CIFAR-10 | 75.3% | 81.4% | 72.5% |
Median validation accuracy, 3 seeds (42–44), demo CNN on CIFAR-10. Baselines: 5 epochs fixed; BNNR: full branch-search pipeline (more compute). Not SOTA — illustrative comparison. Details: benchmarks/README.md · reproduce: benchmarks/run.py → benchmarks/summarize.py.
A fine-grained ResNet18 / Imagewoof benchmark (low-data, from-scratch; 5 seeds, RandAugment + TrivialAugment baselines) is in progress — cheap enough for a free Colab T4, with a one-command reproduce script. See
benchmarks/README.md.
Same CIFAR-10 validation image (test index 127, seed 44): original input plus OptiCAM overlays after three training setups. With crop+flip only, attention scatters toward image edges; RandAugment is more centered but diffuse. After BNNR branch search (ICD + AICD + ChurchNoise), heatmaps concentrate on the vehicle body.
Original → crop+flip · RandAugment (2,9) · BNNR branch search. Illustrative example from the benchmark harness.
Full overlays for 8 fixed val indices: benchmarks/runs/*/xai/ · regenerate figure: python scripts/build_benchmark_xai_readme_asset.py --pick val127
Real metrics from a BNNR training run — branch tree, charts, XAI previews, and dataset insights.
| Overview | Branch Tree | Metrics |
|---|---|---|
 |
 |
 |
| Samples & XAI | Analysis | Dataset Insight |
|---|---|---|
 |
 |
 |
- XAI-driven augmentation (ICD / AICD) — augmentations guided by saliency maps; no other PyTorch toolkit combines explainability and data augmentation this way.
- Auto-augmentation search — iterative branching keeps only augmentations that measurably improve your validation metric.
- Auditable reports — structured JSON reports with metrics, XAI heatmaps, and branch decisions for stakeholders or compliance review.
| Resource | URL |
|---|---|
| Website | bnnr.dev |
| BNNR on LinkedIn | |
| Documentation | docs/README.md |
| Examples | docs/examples.md |
| Roadmap (Q2–Q3 2026) | docs/roadmap.md |
| Colab (classification) | Open in Colab |
| API reference | docs/api_reference.md |
Model analysis (bnnr analyze) |
docs/analyze.md |
| Integrations (Grad-CAM, Ultralytics YOLO) | docs/integrations.md |
| Sample analyze report (live HTML) | raw.githack.com preview |
| GitHub Discussions | Q&A and showcase |
import bnnr
result = bnnr.quick_run(model, train_loader, val_loader)
print(result.best_metrics)For a one-epoch smoke test: bnnr.quick_run(..., m_epochs=1, max_iterations=1).
Advanced: Golden path (BNNRTrainer, custom adapters, detection). API details: api_reference.md.
Install from source, CLI reference, full doc index
git clone https://github.com/bnnr-team/bnnr.git
cd bnnr
(cd dashboard_web && npm ci && npm run build)
pip install -e ".[dev,dashboard]"The PyPI wheel ships the bnnr package only. Runnable scripts (examples/), notebooks, and the documentation tree (docs/) live in this repository.
python3 -m bnnr --help
python3 -m bnnr demo
python3 -m bnnr train --help
python3 -m bnnr analyze --help
python3 -m bnnr report --help
python3 -m bnnr list-datasets
python3 -m bnnr list-augmentations -v
python3 -m bnnr list-presets
python3 -m bnnr dashboard serve --run-dir reports --port 8080
python3 -m bnnr dashboard export --run-dir reports/run_YYYYMMDD_HHMMSS --output exported_dashboard- Getting started
- Python quickstart (
quick_run) - Configuration
- CLI
- Dashboard
- Augmentations
- Detection
- Analyze (standalone diagnostics)
- Examples
- Notebooks
- Artifacts
- Benchmarks
- Citation
- Troubleshooting
- Python
>=3.10 - Core:
torch,torchvision,numpy,typer,pydantic,pyyaml,grad-cam - Dashboard extra:
fastapi,uvicorn,websockets,qrcode
- BNNR software (any feature): cite the software entry (DOI 10.5281/zenodo.20581372; authors: Walo, Morzhak, Zydorczyk, Saczuk).
- ICD or AICD: also cite the method paper (DOI 10.5281/zenodo.20581077).
Full BibTeX: docs/citation.md. Authors and roles: AUTHORS.md. GitHub: Cite this repository (CITATION.cff).
MIT License — use BNNR freely in research, production, and commercial projects.
If BNNR saved you time debugging a vision model, consider starring the repo — it helps others discover the project.