Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added docker-multi-stage builds #10832

Merged
merged 1 commit into from
Dec 22, 2024
Merged

Conversation

rudiservo
Copy link
Contributor

Added multi-stage dockerfile builds, improved total build time to under 2 hours, added Vulkan and Full-intel.

Updated rocm dockerfile.

Hopefully it will all work without any problems.

@rudiservo rudiservo requested a review from ngxson as a code owner December 14, 2024 20:39
@github-actions github-actions bot added the devops improvements to build systems and github actions label Dec 14, 2024
Copy link
Collaborator

@ngxson ngxson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also fix coding style as reported by editorconfig workflow

.devops/cuda.Dockerfile Outdated Show resolved Hide resolved
.devops/musa.Dockerfile Outdated Show resolved Hide resolved
.github/workflows/docker.yml Outdated Show resolved Hide resolved
@rudiservo rudiservo force-pushed the docker-multi-stage branch 10 times, most recently from bfdf494 to 72d0847 Compare December 15, 2024 02:31
@rudiservo
Copy link
Contributor Author

@ngxson I am really sorry about the spam, starting to use act.
Just a slight issue with caching, will report when fixed.

@rudiservo
Copy link
Contributor Author

rudiservo commented Dec 15, 2024

@ngxson ok figured out the cache, it will be using github cache, still a bit experimental but it works best at this point in time.

In the dockerfiles there is a

cp build/bin/* .

can this be changed to a mv? It would same some space.

Or I can just copy this from the cpu dockerfile

COPY --from=build /app/build/bin/ /app/
COPY --from=build /applib/ /app/
COPY --from=build /app/convert_hf_to_gguf.py /app/
COPY --from=build /app/gguf-py /app/gguf-py

instead of copying the full /app in the full versions.

.devops/musa.Dockerfile Show resolved Hide resolved
.github/workflows/docker.yml Show resolved Hide resolved
@ngxson
Copy link
Collaborator

ngxson commented Dec 15, 2024

In the dockerfiles there is a

cp build/bin/* .

can this be changed to a mv? It would same some space.

Yes I think we can, I don't see any problem with this as we never run cmake after that (cc @slaren too, in case I'm missing something)

I'll test on cpu + CUDA in the next few days when I come back from vacation

@slaren
Copy link
Collaborator

slaren commented Dec 15, 2024

Yes, I don't see why the cp couldn't be replaced with a mv. Same for the find -exec cp that is used to copy the .so files. Although all of this should probably be replaced with cmake --install in the future.

@rudiservo
Copy link
Contributor Author

rudiservo commented Dec 15, 2024

@slaren agree, but I would be happy to know what do I need to copy for the full image to work instead of the complete /app folder just to try and keep these images has small has possible for now.

From what I can tell it's the python scripts, the .so and the build/bin.
I will make the appropriate changes if that is ok?

@slaren
Copy link
Collaborator

slaren commented Dec 16, 2024

I already tried to copy as little as possible for the full image in the full.Dockerfile image. The same should work for the other backend images.

@rudiservo rudiservo force-pushed the docker-multi-stage branch 8 times, most recently from 735b9b0 to bf1caab Compare December 17, 2024 00:06
@rudiservo
Copy link
Contributor Author

I normalized all docker images, improved the code a bit, I think everything is ok.
Total build time for 15 images is now at ~1h50.

@rudiservo
Copy link
Contributor Author

It's pushing multiple untaged images, checking.

@rudiservo
Copy link
Contributor Author

Fixed, added provenance to build-and-push.

@ngxson
Copy link
Collaborator

ngxson commented Dec 22, 2024

The tag name for master branch is incorrect. Expected to be full-bXXXX but with this PR, we get fullbXXXX

image

How I tested it: https://github.com/ngxson/llama.cpp-test-ci/pkgs/container/llama.cpp-test-ci/versions

@ngxson
Copy link
Collaborator

ngxson commented Dec 22, 2024

Other than that, I can confirm that the built image is working as expected. Tested with CPU and CUDA images.

@rudiservo
Copy link
Contributor Author

@ngxson Put up a fix, I think the versioning is ok now.

Copy link
Collaborator

@ngxson ngxson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working as expected now, thanks.

Side note, this change may affect users who rely on the old Dockerfile to specifically build server or cli (cmake with --target). In case someone ask, we will find a solution later on.

@ngxson ngxson merged commit 7c0e285 into ggerganov:master Dec 22, 2024
2 checks passed
@rudiservo
Copy link
Contributor Author

More than happy to help, if any issues arise call me up.

@rudiservo rudiservo deleted the docker-multi-stage branch December 22, 2024 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops improvements to build systems and github actions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants