Skip to content

Commit

Permalink
ggml: update freq_factors comment in rope.c
Browse files Browse the repository at this point in the history
Signed-off-by: Daniel Bevenius <[email protected]>
  • Loading branch information
danbev committed Aug 3, 2024
1 parent d2f5267 commit 7ef211a
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions fundamentals/ggml/src/rope.c
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,9 @@ int main(int argc, char **argv) {
float beta_fast = 32.0f;
// This is a YaRN parameter which I think is named β in the YaRN paper.
float beta_slow = 1.0f;
// RoPE Frequency factors are used with certan models like
// Phi-3-mini-128k-instruct (https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/blob/main/config.json#L27)
// LongRope Frequency factors (freq_factors/rope_scaling) are used with
// certain models like Phi-3-mini-128k-instruct
// (https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/blob/main/config.json#L27).
struct ggml_tensor* freq_factors = NULL;

struct ggml_tensor* s = ggml_rope_ext(ctx,
Expand Down

0 comments on commit 7ef211a

Please sign in to comment.