Skip to content

Conversation

@chraac
Copy link
Contributor

@chraac chraac commented Nov 18, 2025

Fixes some failure on the Hexagon swiglu/silu op and improves correctness of the GGML Hexagon backend.

Summary of changes

  • Add overflow-guarded HVX primitives (hvx_vec_exp_fp32_guard, hvx_vec_inverse_fp32_guard) and use them in the Hexagon backend.
  • Improve NaN/Inf handling in exp, inverse, and related helper func.
  • Fix mistakes in the Hexagon silu / swiglu implementations (including handling of src1 and swapped/split variants).

Before

[SILU] NMSE = 3.457965465 > 0.000000100   SILU(type=f32,ne_a=[128,2,2,2],v=0): �[1;31mFAIL�[0m
[SILU] NMSE = 0.496767445 > 0.000000100   SILU(type=f32,ne_a=[5,7,11,13],v=0): �[1;31mFAIL�[0m
[SWIGLU] NMSE = 3894.832699597 > 0.000000100   SWIGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=0): �[1;31mFAIL�[0m
[SWIGLU] NMSE = 2.032236263 > 0.000000100   SWIGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=0): �[1;31mFAIL�[0m
[SWIGLU] NMSE = 531.844516626 > 0.000000100   SWIGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=1): �[1;31mFAIL�[0m
[SWIGLU] NMSE = 1.988691331 > 0.000000100   SWIGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=1): �[1;31mFAIL�[0m
[SWIGLU] NMSE = 1040.190893229 > 0.000000100   SWIGLU(type=f32,ne_a=[128,2,2,2],v=0,split): �[1;31mFAIL�[0m
[SWIGLU] NMSE = 0.493175916 > 0.000000100   SWIGLU(type=f32,ne_a=[5,7,11,13],v=0,split): �[1;31mFAIL�[0m

After

[SILU] NaN at index 231 (HTP0=-nan CPU=88.449669)   SILU(type=f32,ne_a=[128,2,2,2],v=0): �[1;31mFAIL�[0m
  SILU(type=f32,ne_a=[5,7,11,13],v=0): �[1;32mOK�[0m
[SWIGLU] NaN at index 122 (HTP0=nan CPU=-11446.431641)   SWIGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=0): �[1;31mFAIL�[0m
  SWIGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=0): �[1;32mOK�[0m
[SWIGLU] NMSE = 3.835742624 > 0.000000100   SWIGLU(type=f32,ne_a=[128,2,2,2],v=0,swapped=1): �[1;31mFAIL�[0m
  SWIGLU(type=f32,ne_a=[5,7,11,13],v=0,swapped=1): �[1;32mOK�[0m
[SWIGLU] NaN at index 216 (HTP0=nan CPU=-8444.154297)   SWIGLU(type=f32,ne_a=[128,2,2,2],v=0,split): �[1;31mFAIL�[0m
  SWIGLU(type=f32,ne_a=[5,7,11,13],v=0,split): �[1;32mOK�[0m

@chraac chraac marked this pull request as draft November 18, 2025 03:24
static const float kMaxExp = 88.02f; // log(INF)

const HVX_Vector max_exp = hvx_vec_splat_fp32(kMaxExp);
const HVX_Vector inf = hvx_vec_splat_fp32(kInf);
Copy link
Contributor Author

@chraac chraac Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought we can move this init out of the for loop below.

uint32_t w[VLEN_FP32];
__fp16 fp16[VLEN_FP16];
float fp32[VLEN_FP32];
} __attribute__((aligned(VLEN), packed)) HVX_VectorAlias;
Copy link
Contributor Author

@chraac chraac Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its safe to use the gcc ext since in htp we're using clang.

const float limit = ((const float *) (op_params))[3];

const int nc = (src1_valid) ? ne0 : ne0 / 2;
const int nc = (src1_valid) ? ne00 : ne00 / 2;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks we should use src0->ne[0] here instead of dst->ne[0]

// neg by setting the fp32 sign bit
HVX_Vector mask = Q6_V_vsplat_R(0x80000000);
return Q6_V_vor_VV(v, mask);
return Q6_V_vxor_VV(v, mask);
Copy link
Contributor Author

@chraac chraac Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using xor here to fix when x < 0

@github-actions github-actions bot added the ggml changes relating to the ggml tensor library for machine learning label Nov 18, 2025
@chraac chraac marked this pull request as ready for review November 19, 2025 01:55
Copy link
Collaborator

@max-krasnyansky max-krasnyansky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Thanks for the updates.
This PR also fixes #16854

@max-krasnyansky max-krasnyansky merged commit 21d31e0 into ggml-org:master Nov 20, 2025
72 of 74 checks passed
@chraac
Copy link
Contributor Author

chraac commented Nov 21, 2025

This PR also fixes #16854

Just a heads-up: I've only fixed the opt_path == false path here. hvx_fast_sigmoid_f32 is still tricky because of the manual float exponent math causing NaNs, so I'll tackle that in a separate PR.

@chraac chraac deleted the dev-fix-swiglu branch November 21, 2025 06:02
@max-krasnyansky
Copy link
Collaborator

This PR also fixes #16854

Just a heads-up: I've only fixed the opt_path == false path here. hvx_fast_sigmoid_f32 is still tricky because of the manual float exponent math causing NaNs, so I'll tackle that in a separate PR.

Yep. I saw that other tests are still failing.
But the Qwen3-0.6B output is definitely "fixed", as in it's actually sensible now :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants