Implements SSL-EY

[jameschapman19]

Adding SSL-EY from (https://arxiv.org/abs/2310.01012, https://neurips.cc/virtual/2023/80864)

For the moment I've implemented the simple version but it should in principle plug into multicrop like SWAV.

Will have a stab at running the shorter benchmarks shortly.

[vturrisi]

hey @jameschapman19 thanks for contributing. I'll try to start checking your code soon.

[vturrisi]

You already have the augmentations below, either remove this or move the augmentations to a separate file (both are fine)

[jameschapman19]

Good spot - a heads up I think this is also the case for scripts/pretrain/imagenet-100/vicreg.yaml in that case.

This method should plug in wherever VICReg is used. In many ways the benefit of the loss function is it's got a lot of the nice properties of VICReg but without needing to tune the three loss function parameters.

[vturrisi]

No idea why VICReg is like that haha. I'm fine with having a separate file for the augmentations or having it merged with the main file. Should also prob fix VICReg after that.

[jameschapman19]

@vturrisi - just working through ironing things out perhaps a premature PR I just wanted to get a hang of the CI. Will give you a shout when I'm passing more of these.

[vturrisi]

@jameschapman19 no worries, take your time :) and ping me when you are done

[codecov]

Codecov Report

Merging #378 (9e8ad7e) into main (0eadf3d) will increase coverage by 0.12%.
Report is 1 commits behind head on main.
The diff coverage is 93.75%.

❗ Current head 9e8ad7e differs from pull request most recent head 11eac92. Consider uploading reports for the commit 11eac92 to get more accurate results

Additional details and impacted files

Flag	Coverage Δ		*Carryforward flag
cpu	80.02% <93.75%> (+0.15%)	⬆️
dali	38.81% <ø> (ø)		Carriedforward from 0eadf3d

*This pull request uses carry forward flags. Click here to find out more.

Files	Coverage Δ
solo/losses/__init__.py	100.00% <100.00%> (ø)
solo/methods/__init__.py	100.00% <100.00%> (ø)
solo/methods/ssley.py	100.00% <100.00%> (ø)
solo/losses/ssley.py	75.00% <75.00%> (ø)