Code

Bamba architecture#10810

Code

Draft

gabe-l-hart wants to merge 43 commits intoggerganov:masterggerganov/llama.cpp:masterfrom gabe-l-hart:BambaArchitecturegabe-l-hart/llama.cpp:BambaArchitectureCopy head branch name to clipboard

+1,486-287

Commits

Commits on Aug 21, 2024

• llama : initial Mamba-2 support

  compilade

  committed
  1f0fea7

• ggml : SIMD ggml_ssm_scan for Mamba-2

  Show description for dceff23
  compilade

  committed
  dceff23

• llama : support running Mamba-Codestral-7B-v0.1

  compilade

  committed
  2bfe9de

• llama : fix Mamba-2 conv state saving

  Show description for aff9692
  compilade

  committed
  aff9692

Commits on Aug 22, 2024

• llama : remove unused variable

  compilade

  committed
  e04910d

• llama : add missing break

  compilade

  committed
  fa358e7

• convert_hf : prefer SentencePiece tokenizer for Mamba-2 when present

  Show description for 38913dc
  compilade

  committed
  38913dc

Commits on Sep 18, 2024

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  0e601ca

Commits on Oct 1, 2024

• llama : avoid redundant state copy for Mamba 1 and 2

  compilade

  committed
  273e7a4

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  7d6cb36

Commits on Oct 2, 2024

• metal : attempt to adapt SSM_SCAN for Mamba-2

  compilade

  committed
  2c77d79

• metal : fix SSM_SCAN pipeline scope

  compilade

  committed
  87b97d0

• metal : use log and exp instead of log1pf and expf in SSM_SCAN

  compilade

  committed
  03d0e6e

• metal : remove unused arguments for SSM_SCAN

  Show description for 7a351ab
  compilade

  committed
  7a351ab

• metal : add back n_seqs to SSM_SCAN args

  Show description for 8b15bc6
  compilade

  committed
  8b15bc6

• metal : fix SSM_SCAN state head offset

  compilade

  committed
  5b8ec2b

Commits on Oct 3, 2024

• metal : fix wrong number of tokens per sequence in SSM_SCAN

  compilade

  committed
  62b09b3

Commits on Oct 12, 2024

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  038d958

• ggml : remove unused fast broadcast path in GGML_MUL

  Show description for 805512a
  compilade

  committed
  805512a

Commits on Nov 1, 2024

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  7d16e1b

Commits on Nov 4, 2024

• ggml : avoid multiply by D in GGML_OP_SSM_SCAN

  Show description for 3bc7103
  compilade

  committed
  3bc7103

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  8d8f065

• convert : fix flake8 lint

  compilade

  committed
  b4e9c59

Commits on Nov 25, 2024

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  1ee6c48

Commits on Dec 12, 2024

• feat(jamba): First pass at GGUF conversion for Jamba models

  Show description for 9a68f75
  gabe-l-hart

  committed
  9a68f75

• feat(jamba): Add jamba architecture to llama.cpp enums

  Show description for 246dfdb
  gabe-l-hart

  committed
  246dfdb

• feat(convert): Full pass at hparam conversion

  Show description for e3525e9
  gabe-l-hart

  committed
  e3525e9

• fix(bamba conv): Jamba -> Bamba

  Show description for fd98682
  gabe-l-hart

  committed
  fd98682

• fix(bamba): Jamba->Bamba in llama.cpp

  Show description for 1c1e008
  gabe-l-hart

  committed
  1c1e008

• feat(bamba): hparam parsing in llama.cpp

  Show description for e0af809
  gabe-l-hart

  committed
  e0af809

• fix(bamba conv): Fizes in tensor name and hparam conversion for llama.cpp parsing

  Show description for fd3bb30
  gabe-l-hart

  committed
  fd3bb30

• feat(bamba): Full tensor parsing for bamba

  Show description for 3ee0ae3
  gabe-l-hart

  committed
  3ee0ae3

• fix(bamba conv): Remove chunk size and consolidate head count w/ time step rank

  Show description for dfe8d3d
  gabe-l-hart

  committed
  dfe8d3d

• fix(bamba): Remove ssm_head_count and ssm_chunk_size in llama.cpp

  Show description for 41fc019
  gabe-l-hart

  committed
  41fc019

• feat(bamba): Partially complete work on constructing the forward graph

  Show description for e7b1abb
  gabe-l-hart

  committed
  e7b1abb

• fix: Get n_head_kv per-layer in build_bamba

  Show description for f2478bc
  gabe-l-hart

  committed
  f2478bc

• fix: per-layer recurrent embd_[kv]_s

  Show description for d3a34e0
  gabe-l-hart

  committed
  d3a34e0

• WIP: Partial work towards separate hybrid cache

  Show description for 92653d0
  gabe-l-hart

  committed
  92653d0

• fix: Only allocate kv cache tensors for the appropriate layers in hybrid models

  Show description for 44bf431
  gabe-l-hart

  committed
  44bf431

• feat: Update the logic in llama_decode_internal for kv_hybrid cache

  Show description for 4543ed5
  gabe-l-hart

  committed
  4543ed5

• fix: A number of places where hybrid needs to be handled

  Show description for 204e78f
  gabe-l-hart

  committed
  204e78f

• fix: Remove outdated TODO in convrsion script

  Show description for 97e6ba8
  gabe-l-hart

  committed
  97e6ba8

• fix: Remove unused LLM_KV_ATTENTION_LAYER_COUNT

  Show description for b83e9a6
  gabe-l-hart

  committed
  b83e9a6