Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an ability to switch the build architecture from 'musl' to 'glibc' #8419

Open
chrisdonlan opened this issue Dec 13, 2024 · 0 comments · May be fixed by #8420
Open

Add an ability to switch the build architecture from 'musl' to 'glibc' #8419

chrisdonlan opened this issue Dec 13, 2024 · 0 comments · May be fixed by #8420

Comments

@chrisdonlan
Copy link

We have a use case for lakectl running in glibc based containers. glibc containers are required because we'll be running pytorch.

Although ChatGPT is suggesting pytorch can be installed on alpine by installing glibc on alpine, I am not ready to introduce that complication if it is unnecessary. In particular, I have a feeling leveraging a GPU from alpine w/ pytorch could be nearly impossible, and, even if it is possible, solutions would tend to be unique across GPU versions, driver versions, and glibc/alpine versions.

So, we are going to prefer glibc images as base images. But, we still want to use LakeFS. The main problem is that while musl images work out of the box on arm64, they do not work at all on x86_64 base images. I have a few solutions:

  • wget the correct binaries from github during the build. This is serviceable, but problematic in some contexts.
  • git-lfs store the correct binaries in the project being built
  • artifactory distribute the binaries from our artifactory

Or, build the lakeFS image in our normal workflow. Of all the options, building the LakeFS image in our normal workflow seems to be the lowest touch, most robust, and simplest solution over the next few years.

So the current Dockerfile only builds for alpine in linux-musl-amd64 architecture. It'll have to be generalized with build-args to allow for other platforms, such as glibc, to be used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants