Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

musa: fix aarch64 build #10781

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

musa: fix aarch64 build #10781

wants to merge 1 commit into from
+16 −2

Conversation

BodhiHu
Copy link

@BodhiHu BodhiHu commented Dec 11, 2024 •
edited
Loading

Hi, this PR will:

  1. fix building for MUSA when targeting aarch64;

@github-actions github-actions bot added documentation Improvements or additions to documentation build Compilation issues Nvidia GPU Issues specific to Nvidia GPUs ggml changes relating to the ggml tensor library for machine learning labels Dec 11, 2024
@BodhiHu
Copy link
Author

BodhiHu commented Dec 11, 2024

HI @ggerganov , could you please help review this PR ?

@ggerganov
Copy link
Owner

What errors do you get before this PR?

@BodhiHu
Copy link
Author

BodhiHu commented Dec 12, 2024

Hi,

I got these errors(with the full compile command printed):

image

image

To fix this, had to undefine __ARM_NEON on the cuda code when GGML_USE_MUSA is on:

#ifdef GGML_USE_MUSA
#undef __ARM_NEON
#endif

@slaren
Copy link
Collaborator

slaren commented Dec 12, 2024

The -march etc flags should not be used to build the musa backend. I would try fixing that first.

kkontny
kkontny reviewed Dec 12, 2024
View reviewed changes
ggml/src/ggml-cpu/CMakeLists.txt Outdated
if (${CMAKE_SYSTEM_PROCESSOR} MATCHES "aarch64")
message(STATUS "Linux aarch64 detected")

set(MARCH_FLAGS "-march=armv8.2-a+fp16")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes 2 things:

  1. Every aarch64 CPU supports fp16. I think the code should look for fphp CPU feature.
  2. aarch64 will be at least armv8.2, in fact aarch64 implies only armv8

@BodhiHu
Copy link
Author

BodhiHu commented Dec 16, 2024 •
edited
Loading

Hi @slaren , @kkontny , I had removed the CPU parts and -march on musa side, to just fix the compiler errors.
Not sure if it's a proper fix, or we should wait for musa arm sdk to support NEON :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kkontny kkontny kkontny left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
build Compilation issues documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning Nvidia GPU Issues specific to Nvidia GPUs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@BodhiHu @ggerganov @slaren @kkontny