🚀 [ICLR 2025] Few-Class Arena: A Benchmark for Efficient Selection of Vision Models and Dataset Difficulty Measurement

[bryanbocao]

We are excited to introduce Few-Class Arena (FCA), a unified benchmark with focus on testing efficient image classification models for few classes published in ICLR 2025.

🔍 Why Few-Class Arena?
A wide variety of benchmark datasets with many classes (80-1000) have been created to assist Computer Vision architectural evolution. An increasing number of vision models are evaluated with these many-class datasets. However, real-world applications often involve substantially fewer classes of interest (2-10). This gap between many and few classes makes it difficult to predict performance of the few-class applications using models trained on the available many-class datasets. To date, little has been offered to evaluate models in this Few-Class Regime.

📄 Paper:
https://openreview.net/forum?id=2ET561DyPe

🙏 Acknowledgments

We sincerely thank the OpenMMLab community and all contributors to MMPretrain for building a robust, modular, and high-quality training framework that serves the foundation of the Few-Class Arena tool.

🔗 FCA repository:
https://github.com/bryanbocao/fca

🔗 FCA maintainer:
https://github.com/bryanbocao

Citation

@article{cao2024few,
  title={Few-Class Arena: A Benchmark for Efficient Selection of Vision Models and Dataset Difficulty Measurement},
  author={Cao, Bryan Bo and O'Gorman, Lawrence and Coss, Michael and Jain, Shubham},
  journal={arXiv preprint arXiv:2411.01099},
  year={2024}
}

@misc{fca2025,
  author = {Bryan Bo Cao},
  title = {Few-Class Arena (FCA)},
  year = {2025},
  howpublished = {\url{https://github.com/bryanbocao/fca}},
  note = {Accessed: 2025-03-01}
}

Note change the access date.