Code

Speculative sampling#17

Code

Open

daviswer wants to merge 10 commits intomainfoundation-model-stack/fms-extras:mainfrom speculative_samplefoundation-model-stack/fms-extras:speculative_sampleCopy head branch name to clipboard

+108-11

Commits

Commits on Apr 11, 2024

• Consolidate best guess and correct tokens

  daviswer

  committed
  3ed5a89

  Copy full SHA for 3ed5a89

• Add candidate-consistent next_val sampling

  daviswer

  committed
  1ac6623

  Copy full SHA for 1ac6623

• fixed issue with top_k in sampling; fixed type hint for temperature; added args to script for sampling

  JRosenkranz

  committed
  be7e0cc

  Copy full SHA for be7e0cc

• Add docstring for generate_targets

  daviswer

  committed
  96fdc7d

  Copy full SHA for 96fdc7d

• Simpler/faster consistent sampling

  daviswer

  committed
  ea646a2

  Copy full SHA for ea646a2

Commits on Apr 12, 2024

• Vectorized do_sample in generate_targets

  daviswer

  committed
  f6d96f6

  Copy full SHA for f6d96f6

Commits on Apr 29, 2024

• Merge branch 'main' into speculative_sample

  daviswer

  authored
  82a591f

  Copy full SHA for 82a591f

Commits on May 13, 2024

• Cleaner sample/nosample masking

  daviswer

  committed
  6552516

  Copy full SHA for 6552516

Commits on Jun 17, 2024

• Fix do_sample typing issues

  daviswer

  authored
  2f47c70

  Copy full SHA for 2f47c70

• Linting

  daviswer

  authored
  bc75432

  Copy full SHA for bc75432