[ENH] online update capability for probabilistic regressors

[fkiraly]

Adds framework support for online update capability for probabilistic regressors, and a simple composite strategy that refits on all data, for testing the framework. Closes #463

Contains:

• extension of the regressor and survival base class with a potential update / _update method for batch updates
• addition of a tag capability:online for respective estimators
• addition of a composite OnlineRefit that adds the capability:online tag and refits the regressor upon all data seen so far. This is a separate estimator to avoid that all estimators remember (and clutter self) with the data
• a similar composite OnlineDontRefit that turns off online capability
• a specific test case for online updates, in TestAllRegressors

[fkiraly]

FYI @simon-hirsch, @BerriJ - this extends the framework to add online methods :-)

[simon-hirsch]

Looks generally quite cool to me 👍

Generally, I think putting the "remember old data and fit on the union of new data and old data" strategy in a separate estimator is a good thing, as it potentially dangerous wrt to the disc space an estimator saved with pickle / joblib / ... takes up and will slow down the storing/loading of models.

For testing, you'd might want to use a TimeSeriesSplit instead of the random test_train_split. For exact online learning methods one could even test whether the update leads indeed to the repeated batch fit, this is of course more tricky for approximate methods like SGD.

[fkiraly]

For testing, you'd might want to use a TimeSeriesSplit instead of the random test_train_split.

This is just testing the interface, and it should not matter imo for the test.

Regarding the "conceptual model", unlike sklearn we do not assume or test that regressors have an exchangable behaviour with respect to sample index. Once we get the first examples of regressors that assume ordering or other types of non-exchangeability, we could simply distinguish them by tag.

[fkiraly]

Generally, I think putting the "remember old data and fit on the union of new data and old data" strategy in a separate estimator is a good thing, as it potentially dangerous wrt to the disc space an estimator saved with pickle / joblib / ... takes up and will slow down the storing/loading of models.

Agreed, I think it is an issue with sktime forecasters already - there, storing seems more important since there's no "y from X" pairing, though I'd also like to get rid of it as much as possible.