[BUG] Fixed subsampling in highly imbalances datasets giving subsamples with only a single class

[ferewi]

Reference Issues/PRs

Fixes #1726

What does this implement/fix? Explain your changes.

Added new attribute 'max_subsamples' to subsample multiple times in case of unbalanced datasets giving subsamples with only one class. This caused TDE and HC2 to fail.
If a subsample with only a sinlge class is found, subsampling is repeated until the number of classes in the subsample is >1 or until max_subsamples is reached. In the latter case an AttributeError is raised.

Does your contribution introduce a new dependency? If yes, which one?

No

Any other comments?

In the Issue I proposed two ideas how to fix this and I went for a modified option 1 that does not lead to potentially infinite loops as it is more than 3 times faster compared to option 2 (StratifiedShuffleSplit).

PR checklist

@all-contributors add @ferewi for code

For all contributions

• I've added myself to the list of contributors. Alternatively, you can use the @all-contributors bot to do this for you.
• The PR title starts with either [ENH], [MNT], [DOC], [BUG], [REF], [DEP] or [GOV] indicating whether the PR topic is related to enhancement, maintenance, documentation, bugs, refactoring, deprecation or governance.

For new estimators and functions

does not apply

• I've added the estimator to the online API documentation.
• (OPTIONAL) I've added myself as a __maintainer__ at the top of relevant files and want to be contacted regarding its maintenance. Unmaintained files may be removed. This is for the full file, and you should not add yourself if you are just making minor changes or do not want to help maintain its contents.

For developers with write access

does not apply

• (OPTIONAL) I've updated aeon's CODEOWNERS to receive notifications about future changes to these files.

[aeon-actions-bot]

Thank you for contributing to aeon

I have added the following labels to this PR based on the title: [ $\color{#d73a4a}{\textsf{bug}}$ ].
I have added the following labels to this PR based on the changes made: [ $\color{#BCAE15}{\textsf{classification}}$ ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

• Run pre-commit checks for all files
• Run mypy typecheck tests
• Run all pytest tests and configurations
• Run all notebook example tests
• Run numba-disabled codecov tests
• Stop automatic pre-commit fixes (always disabled for drafts)
• Disable numba cache loading
• Push an empty commit to re-run CI checks

[MatthewMiddlehurst]

Is there a situation where you wouldn't want to continue until there is a viable subsample? I would just use a while loop and continue until you do. No parameter needed then.

The only time this will backfire is if there is only 1 class in the input, but we should catch that higher up in input verification IMO.

[TonyBagnall]

this is good but we need a little time to discuss, I will put it on the agenda for the dev meeting on friday

[ferewi]

I was a bit caught up last week hence my late reply. @MatthewMiddlehurst I think your're right, the new parameter is actually not needed. I was thinking that given a large unbalanced dataset, the chance for drawing a subsample that contains just one class is high, but given the subsample size of 70% this chance is around 0.5 in the worst case. So just resampling until a valid sample is obtained should be fine.

[MatthewMiddlehurst]

LGTM, thanks.

[MatthewMiddlehurst]

@all-contributors add @ferewi for bug

[allcontributors]

@MatthewMiddlehurst

I've put up a pull request to add @ferewi! 🎉