ENH: Enable accelerated linear regression for overdetermined systems #2083
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
As a follow-up from oneDAL PR: oneapi-src/oneDAL#2930
The PR above enables fitting linear regressions to overdetermined systems, both in regular regression and multi-output regression. Currently, the
sklearnex
estimators fall back toscikit-learn
for overdetermined systems:scikit-learn-intelex/sklearnex/linear_model/linear.py
Line 180 in 2fccf44
But after the PR above, those should be supported in oneDAL too. Hence, this PR enables using oneDAL for those situations and adds tests for them.
One thing to note though: the mechanism in oneDAL is implemented as a fallback from Cholesky - that is, it will first perform a Cholesky factorization, and if that fails, will then use the fallback solver which can handle overdetermined systems. But if e.g. there are more columns that rows, we'll already know beforehand that Cholesky will invariably fail, and the procedure could be made faster if such kind of solver could be requested straight away from oneDAL. Implementation of it as a selectable solver is left for future PRs.
Note that since the oneDAL PR hasn't been merged at the time of writing, this PR will of course end up failing tests, but will keep it here in the meantime.
Checklist to comply with before moving PR from draft:
PR completeness and readability
Testing
Performance