whisper-burn

Speech-to-text with OpenAI Whisper, written in pure Rust on top of the Burn deep-learning framework. One self-contained command-line tool: no Python, no ffmpeg, no PyTorch.

• Reads WAV, MP3, FLAC, OGG/Vorbis, AAC/M4A, ALAC directly (any sample rate, any channel count).
• Downloads models for you: whisper-burn model base.
• Transcribes in 99+ languages, or translates speech to English.
• Splits long recordings into 30-second pieces at pauses, not in the middle of a phrase.
• Runs on the CPU. Built-in benchmark and self-check commands.

$ whisper-burn transcribe speech.mp3 --language en
We knew the world would not be the same. Few people laughed, few people cried, ...

Contents

• Quick start
• Requirements and build
• Command reference
  • model · transcribe · convert · mel · bench · dump
• Models
• Performance
• How it works
• Project layout
• Development
• Limitations
• Troubleshooting
• Acknowledgements

Quick start

# 1. build (first build takes several minutes)
cargo build --release

# 2. download a model into ./whisper-base
cargo run --release -- model base

# 3. transcribe
cargo run --release -- transcribe speech.mp3 --language en

Everything after -- is passed to the program. You can also run the compiled binary directly:

.\target\release\whisper-burn.exe transcribe speech.mp3

Models are stored in the current directory (./whisper-<name>), so run the tool from the folder where you installed them.

Requirements and build

Requirement	Notes
Rust toolchain	A recent stable Rust (1.85 or newer is recommended). Install from rustup.rs.
C compiler	The tokenizers crate builds a small C library. On Windows install Visual Studio Build Tools ("Desktop development with C++"); on Linux build-essential.
Internet	Only for the model command (downloads from Hugging Face). Transcription itself is fully offline.

git clone https://github.com/<your-username>/whisper-burn.git
cd whisper-burn
cargo build --release

The binary is target/release/whisper-burn (.exe on Windows).

HTTPS downloads use the operating system's TLS stack (SChannel on Windows) via native-tls. This avoids the ring crate, which needs an extra C toolchain setup and fails to build with some recent MSVC versions.

Command reference

whisper-burn <command> [options]

Command	What it does
model [name] [--yes]	List or download Whisper models
transcribe <audio> [options]	Speech to text (or translation to English)
convert <in> <out.wav> [options]	Convert any supported audio file to a mono WAV
mel <audio> [--n-mels N]	Compute the log-mel spectrogram and print statistics
bench [--model DIR]	Micro-benchmarks and per-operation timing
dump [options]	Numerical self-checks of the encoder/decoder

Running the program without a valid command prints the usage text.

model — download and list models

whisper-burn model                 # show available and installed models
whisper-burn model base            # download openai/whisper-base into ./whisper-base
whisper-burn model small --yes     # same for small, without asking questions

Option	Meaning
tiny · base · small	Which model to install. No name = just list.
--yes, -y	Do not ask for confirmation before deleting another installed model.

Behaviour:

• Files come from the Hugging Face repository openai/whisper-<name>: model.safetensors, tokenizer.json and (if present) generation_config.json.
• One model at a time. If another model folder exists (for example whisper-tiny while you install base), you are asked: Found whisper-tiny. Delete it and install whisper-base? [y/N]
• Safe replacement. The new model is downloaded into whisper-base.part first. The old folder is deleted only after every file has arrived, so a failed or interrupted download never destroys the model you already have.
• If the requested model is already installed, nothing is downloaded.
• In a non-interactive session (script, CI) without --yes the command stops with an error instead of deleting anything silently.

transcribe — speech to text

whisper-burn transcribe <audio> [--model DIR] [--language xx] [--translate] [--max-tokens N] [-v]

Option	Default	Meaning
<audio>	required	Audio file. Must be the first argument after transcribe.
--model DIR	installed whisper-* folder in the current directory	Model folder containing model.safetensors and tokenizer.json.
--language xx	auto-detect	Language code such as en, ru, de, fr, ja. Multilingual models only.
--translate	off	Translate the speech into English instead of transcribing it. Multilingual models only.
--max-tokens N	224	Maximum number of generated tokens per 30-second chunk.
-v, --verbose	off	Print chunk boundaries, timings, the detected language and the text of every chunk.

Examples:

# transcribe, language detected automatically
whisper-burn transcribe interview.mp3

# force English (more reliable on noisy or old recordings)
whisper-burn transcribe speech.mp3 --language en

# Russian speech -> English text
whisper-burn transcribe talk.m4a --language ru --translate

# see how the audio was chunked and how long each stage took
whisper-burn transcribe speech.mp3 --language en -v

# use a specific model folder
whisper-burn transcribe speech.mp3 --model D:\models\whisper-small

The text goes to stdout, progress and timing messages go to stderr, so redirection saves only the text. In Windows PowerShell 5.1 use Out-File to get UTF-8:

whisper-burn transcribe speech.mp3 --language en | Out-File -Encoding utf8 speech.txt

How long audio is handled: the file is decoded, mixed to mono and resampled to 16 kHz. It is then cut into pieces of at most 30 seconds. Each cut is placed in the quietest 100 ms window of the last 5 seconds before the 30-second limit, so chunks usually end between words. With -v you can see the boundaries:

[chunk 0] 0.0 s .. 27.4 s
[chunk 1] 27.4 s .. 53.4 s

The language is detected once, on the first non-silent chunk, and reused for the rest of the file.

convert — convert audio to WAV

whisper-burn convert <in> <out.wav> [--rate 16000] [--bits 16|24|32f] [--normalize]

Option	Default	Meaning
--rate N	16000	Output sample rate in Hz (1000 – 384000). Uses a band-limited polyphase resampler (Kaiser-windowed sinc, anti-aliased).
--bits	16	16 or 24 bit PCM, or 32f for 32-bit float.
--normalize	off	Scale so the loudest sample reaches about −1 dBFS. Silence is left untouched.

The output is always mono. Missing output folders are created.

whisper-burn convert speech.mp3 speech.wav
whisper-burn convert music.flac out\music_48k.wav --rate 48000 --bits 24 --normalize

mel — log-mel spectrogram stats

whisper-burn mel <audio> [--n-mels 80|128]

Computes the log-mel spectrogram of the first 30 seconds and prints its shape ([n_mels, 3000]), minimum, maximum, mean and the computation time. Use --n-mels 128 for models that need 128 mel bands (large-v3 and turbo); transcribe picks the right number automatically from the model.

bench — performance breakdown

whisper-burn bench                              # micro-benchmarks only
whisper-burn bench --model whisper-small        # + full encoder/decoder breakdown

Without --model it measures matrix multiplication (GFLOPS), fused vs. manual attention, GELU, element-wise add and the output-projection memory bandwidth. With --model DIR it also times the full encoder (per operation: convolutions, layer norm, K/V projections, attention, MLP, residuals) and the decoder (per token: embeddings, self-attention, cross-attention, MLP, logits). Close heavy programs first for honest numbers.

dump — correctness self-checks

whisper-burn dump [--model DIR] (--wav audio | --mel mel.bin) [--tokens 1,2,3] [--out out.bin]

Runs the encoder and the decoder in three different ways and checks that they agree:

1. encoder output is finite;
2. the argmax computed on the device equals the argmax of the full logits on the host;
3. a -inf logit bias forbids exactly the chosen token;
4. incremental decoding with the KV cache equals one-shot decoding (max difference < 1e-3).

Each check prints OK or MISMATCH. --tokens sets the prompt token ids (default: built from the model's tokenizer: start-of-transcript, language, task, no-timestamps). --out writes [encoder output | logits one-shot | logits incremental] as little-endian f32.

Models

Name	Install	Notes
tiny	whisper-burn model tiny	Smallest and fastest, least accurate.
base	whisper-burn model base	Good speed/quality balance for clean speech.
small	whisper-burn model small	Noticeably better on noisy audio, accents, rare words. About 3× slower than base.

The default models are multilingual. Models are read in the Hugging Face safetensors layout (F32, F16 and BF16 weights are supported). Other Whisper sizes in this layout can be used by pointing --model at the folder; the model command only manages the three sizes above. English-only (.en) models work too, but reject --language and --translate.

Model weights are released by OpenAI under the MIT license and downloaded from Hugging Face (openai/whisper-*). Model folders are large (the small model is close to 1 GB) and are not part of this repository.

Performance

Measured with the commands in this README, release build, Burn CPU backend, on:

CPU	AMD Ryzen 5 3500X (6 cores / 6 threads)
OS	Windows, release build (cargo run --release)
Test clip	53.4 s of English speech, MP3, 44.1 kHz stereo (decoded and resampled to 16 kHz mono first)

End-to-end transcription of the 53.4 s clip (transcribe ... -v):

Model	Total time	Speed vs. real time	Encoder per 30 s chunk	Decoder	Model load
base	2.5 s	≈ 21× faster	≈ 0.7 s	≈ 10.8 ms / token	0.5 s
small	7.3 s	≈ 7× faster	≈ 2.1 s	≈ 31 ms / token	1.5 s

tiny has not been measured on this machine. Extrapolated to a one-hour recording, base would take roughly 3 minutes and small roughly 8 minutes (an estimate from the numbers above, not a measurement).

Things worth knowing:

• Whisper always processes a full 30-second window. A 5-second clip costs almost as much as a 30-second one; long files are where the speed-up over real time shows.
• Numbers vary with background load, CPU boost and RAM speed.

Where the time goes (small, bench --model whisper-small):

Stage	Time	Share
Encoder, per 30 s chunk	≈ 2.2 s
– MLP	1380 ms	63 %
– attention (projections + attention)	589 ms	27 %
– K/V projections	120 ms	5 %
– convolutions	58 ms	3 %
– layer norm / residuals	59 ms	3 %
Decoder, per token	≈ 31 ms
– MLP	11.6 ms	37 %
– cross-attention	7.5 ms	24 %
– logits (vocabulary projection)	6.7 ms	21 %
– self-attention	5.7 ms	18 %

Micro-benchmarks on the same CPU (tiny-sized shapes): matrix multiplication reaches about 273 GFLOPS; Burn's fused attention takes 21 ms vs. 43 ms for the manual matmul + softmax version; the output projection streams at about 27 GB/s, which is why decoding is limited by memory bandwidth rather than by arithmetic.

Reproduce:

cargo run --release -- transcribe speech.mp3 --language en -v
cargo run --release -- bench --model whisper-small

How it works

audio file ──► decode (Symphonia) ──► mono ──► resample to 16 kHz (polyphase sinc)
          ──► split at pauses into ≤ 30 s chunks
          ──► log-mel spectrogram (80/128 bands, same recipe as the reference)
          ──► encoder (2 convolutions + transformer, Burn) ──► [1500, d_model]
          ──► decoder (greedy, KV cache, cross-attention) ──► tokens ──► text

• Decoding is greedy. Non-speech, timestamp, language and special tokens are suppressed (plus the suppress_tokens list from generation_config.json); an empty or blank first token is forbidden.
• Decoder speed-ups: cross-attention keys/values are computed once per chunk; self-attention uses a KV cache; the argmax runs on the device so only one integer per token is read back.
• Mel spectrogram follows the reference implementation: reflect padding, periodic Hann window, n_fft = 400, hop 160, Slaney mel filters, the last STFT frame dropped, dynamic range clamped to 8 decades.
• Weights are read by a small built-in safetensors reader (no extra crate).

Project layout

src/
  main.rs        command-line interface (all commands)
  download.rs    model manager: Hugging Face download, safe replacement
  audio.rs       decoding, resampling, WAV writing
  mel.rs         log-mel spectrogram
  weights.rs     safetensors reader (F32 / F16 / BF16)
  model.rs       Whisper encoder/decoder on Burn
  transcribe.rs  chunking at pauses, language detection, greedy decoding

Development

cargo test --release       # unit tests
cargo run --release -- dump --wav speech.mp3   # numerical self-checks with a real model

The unit tests cover the resampler (in-band tone kept, out-of-band tone removed, odd rates), WAV round trips, the FFT against a naive DFT, the mel filter bank, the safetensors reader, the model folder manager and the chunking logic.

Limitations

• CPU only. No GPU backend is wired up yet.
• No timestamps or subtitles. Output is plain text.
• Greedy decoding only. There is no temperature fallback, so on silence or noise Whisper can "hallucinate" repeated phrases.
• No context between chunks. Chunks are cut at pauses, but the previous text is not fed to the next chunk, so a sentence can still be split across a boundary.
• One language per file. The language is detected on the first chunk.
• Opus (.opus, many .webm files) is not supported; convert to MP3/WAV first.
• Memory. The whole model.safetensors is read into memory and converted to f32; large models need a lot of RAM.
• Tested with the base and small models.

Troubleshooting

Problem	Solution
no model installed: run whisper-burn model tiny ...	Run whisper-burn model base from the folder where you want the model, or pass --model DIR.
unsupported audio codec ... Opus is not	Convert the file: ffmpeg -i in.opus out.mp3, then transcribe the result.
Garbled characters in the saved text file (Windows PowerShell)	Use | Out-File -Encoding utf8 file.txt instead of >.
Model download fails halfway	Run the command again. The old model is untouched; the partial download is discarded.
Wrong language detected / odd text on old recordings	Pass --language xx explicitly, or use a bigger model (small).
Build error about ring / cl.exe	The project avoids ring by using native-tls; make sure ureq is declared with default-features = false, features = ["native-tls"].
Sentences repeat or text appears over silence	Typical Whisper hallucination on silence/noise; trim the silence or try a bigger model.

Acknowledgements

• OpenAI Whisper — the model and the reference recipe (Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision).
• Burn — deep-learning framework.
• Symphonia — audio decoding.
• Hugging Face tokenizers and the openai/whisper-* model repositories.