Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add scikit-learn #696

Closed
wants to merge 5 commits into from
Closed

Conversation

Lash-L
Copy link

@Lash-L Lash-L commented Jun 5, 2024

For a custom component i'm working on/ thinking about releasing, I need scikit-learn to be wheeled for alpine.

This is an issue others have had in the past: https://community.home-assistant.io/t/how-to-use-scikit-learn-with-a-custom-intigration/536939

home-assistant/operating-system#3040

scikit-learn has decided as of a few months ago the do not plan to release a musllinux wheel scikit-learn/scikit-learn#27004 (comment)

I can rebuild the model I want to use from scratch, but I would prefer to just be able to use a robust and optimized model like what scikit-learn has. I am trying to use a decision tree inducer and seemingly most other packages out there either a) aren't as good as scikit-learn by a lot or b) use scikit-learn under the hood. If this isn't ideal for any reason, that's okay, just let me know.

Thanks!

Copy link
Member

@frenck frenck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi there @Lash-L 👋

The CI doens't pass. Can you take a look?

Thanks! 👍

../Frenck

@home-assistant
Copy link

home-assistant bot commented Jun 9, 2024

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

@home-assistant home-assistant bot marked this pull request as draft June 9, 2024 14:47
@Lash-L
Copy link
Author

Lash-L commented Jun 9, 2024

Seems as if it wants a Fortran compiler too. Nothing is ever easy!

../meson.build:78:0: ERROR: Unknown compiler(s): [['gfortran'], ['flang'], ['nvfortran'], ['pgfortran'], ['ifort'], ['ifx'], ['g95']]

I suppose I need to figure out what Fortran compiler is best across different archs and add it - would that go here?

apk: "libffi-dev;rrdtool-dev;geos-dev;opus-dev;libvpx-dev;ffmpeg-dev;libsrtp-dev;libressl-dev"

Is that something fine to do?

@frenck
Copy link
Member

frenck commented Jun 9, 2024

Sure, extra build dependencies can be added if needed 👍

@Lash-L Lash-L marked this pull request as ready for review June 9, 2024 17:10
@home-assistant home-assistant bot requested a review from frenck June 9, 2024 17:10
@Lash-L
Copy link
Author

Lash-L commented Jun 9, 2024

We can see if that does the trick

@frenck frenck marked this pull request as draft June 9, 2024 18:25
@frenck
Copy link
Member

frenck commented Jun 9, 2024

CI failing. Please wait with marking a PR ready for review, until the CI passes.

../Frenck

@Lash-L
Copy link
Author

Lash-L commented Jun 9, 2024

Sorry about that, I couldn't tell if the checks would pass as I haven't committed to this repo before so I marked it as ready in a sense of ready for the ci to run. Will run ci locally on my branch until I figure it out 👍🏻

@Lash-L
Copy link
Author

Lash-L commented Jun 9, 2024

Apparently cannot run the ci on a local branch, but I will test with a docker container running alpine later 👍

@Lash-L
Copy link
Author

Lash-L commented Jun 10, 2024

Okay - it seems to work locally for me, I think it is ready for a ci run here!

@frenck
Copy link
Member

frenck commented Jun 10, 2024

Seems like i386/amd64 will pass now, the ARM-based ones do fail at this point.

@Lash-L
Copy link
Author

Lash-L commented Jun 11, 2024

Odd - I am able to install xsimd and openblas on my rpi running HA and Alpine. I'll have to experiment some more and try to figure it out

@xkns
Copy link

xkns commented Jun 14, 2024

I'm also very interested in this and am happy to help. If there is anything I can do please let me know.

@Lash-L
Copy link
Author

Lash-L commented Jun 14, 2024

I'm out of the country rn but will continue trying to figure this out when I get back

@Lash-L
Copy link
Author

Lash-L commented Jun 29, 2024

Running docker commands directly and it succeeds on amd64, running another but it is going slow and i'm going to bed, but i figured i'd push the update and if you had a chance, a ci run will tell us for sure if it is working or not. If you don't have a chance, I'll try again on some other types, next time i have some free time

@Lash-L
Copy link
Author

Lash-L commented Jun 30, 2024

Successful on a second arch, so this is ready for a CI run

edit: Please ignore - I found that I could run the ci myself on my fork if I editted the CI and set it to test=True.

It takes hours for some archs for some reason, will continue to research

@Lash-L
Copy link
Author

Lash-L commented Jul 1, 2024

I'm going to close this as I believe it is not a feasible task unfortunately. scikit-learn depends on scipy and scipy's installation takes a very long time on some of the alpine archs (aarch64, armhf, armv7) Scipy provides musllinux wheels for x86-64 (https://pypi.org/project/scipy/#files). But when wheels are not provided - it can take up to 5/6 hours to install, which is not feasible: https://github.com/Lash-L/wheels-custom-integrations/actions/runs/9735326489

Perhaps someone else can look into this in the future, I'd be happy to share what I know

@Lash-L Lash-L closed this Jul 1, 2024
@xkns
Copy link

xkns commented Jul 1, 2024

It's never easy, is it?

Anyways, thanks for giving it a shot.

Guess one could package the necessary sklearn source files manually with one's custom component, as long as there is no redistribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants