Issues · lucidrains/rotary-embedding-torch · GitHub

Labels Milestones New issue

Avoid hard-coding autocast device parameter in rotary_embedding_torch.py

#46

· ZRICHARD9527 opened

on Dec 12, 2025

Error when dimension is odd number

#45

· caioartus opened

on Oct 23, 2025

How to use in stream inference?

#40

· haha010508 opened

on Dec 20, 2024

Was inspired by the paper you posted but i think it can be taken further by expanding the idea to multiple attention types (not unlike the paper)

#39

· sine2pi opened

on Nov 28, 2024

Slower than absolute positional embeddings?

#36

· umarbutler opened

on Sep 20, 2024

Explicit casting instead of autocasting

#35

· lminer opened

on Sep 12, 2024

Fine-tuning Axial RoPE with frequency scaling?

#32

· tasansal opened

on Aug 26, 2024

RoPE embeddings

#30

· PRamoneda opened

on Aug 13, 2024

Request for YaRN

#29

· VarunGumma opened

on Aug 3, 2024

RoPE-Mixed: Improvement over Axial for n-D

#25

· tasansal opened

on Jun 27, 2024

`torch.cat` failes in `apply_rotary_emb` when `freqs.shape[-1] == t.shape[-1]`, and `start_index = 0`

#20

· mattaltberg opened

on Feb 29, 2024

Model hangs on eval

#15

· GarrettMerz opened

on Dec 13, 2023