Skip to content

Conversation

fira7s
Copy link
Collaborator

@fira7s fira7s commented Apr 9, 2025

  • Support for multi-dataset OOD eval in classification routine
  • Each datamodule should define near and far OOD datasets to be used by default
  • According to args defined in datamodule (OOD and shift), routine is automatically configured Not possible
  • Add SOTA OOD eval techniques from OpenOOD
  • fix ugly output + add id accuracy using the ood criteria Not needed for now
  • fix imagent 1k eval shift
  • set ood datasets names in output table when using custom datasets
  • add ood datasets to huggingface
  • Add imagenet 200
  • hyperparam search for some ood postprocessors
  • Fix Mnist perf
  • Add ood tutorial
  • fix docstrings
  • fill coverage

@o-laurent
Copy link
Contributor

@fira7s you can install the pre-commits with pre-commit install to avoid linting errors in the CI 🙂

@fira7s
Copy link
Collaborator Author

fira7s commented Apr 10, 2025

My bad i knew it would cause the error but i ignored it @o-laurent i will be also changing more stuff in the routine and datamodules so i may need your opinion after it's done i hope my changes won't break other parts of the library.

@fira7s fira7s requested a review from o-laurent April 10, 2025 07:44
@o-laurent
Copy link
Contributor

Don't worry @fira7s , it was just a piece of friendly advice! I'll have a look, thanks!

@alafage alafage changed the title Multi-dataset General ood eval ✨ Multi-dataset General ood eval May 3, 2025
@o-laurent
Copy link
Contributor

Thanks for your work @fira7s
We should put all the configuration files on HuggingFace 😉

@fira7s
Copy link
Collaborator Author

fira7s commented May 23, 2025

@o-laurent Yes indeed, i'm working on merging this into dev right now, i will put all config files on huggingface once i'm finished.

@o-laurent
Copy link
Contributor

(NB: We'll need to squash the PR)

@o-laurent
Copy link
Contributor

o-laurent commented Jul 31, 2025

@fira7s can you extract your last modifications and put them in a separate PR? I can do it and put you as "commit co-author", if you prefer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants