Skip to content

higgs-audio: crate skeleton + config/weights + backbone forward + parity gate #408

Description

@ywh555hhh

Summary

Sub-issue for #395. Adds the openinfer-higgs-audio crate: config parsing, weight
name mapping, backbone forward (text-only), and a parity gate against golden logits.

See PR #409 for the implementation.

Scope

395.1 — Crate skeleton + feature wiring

  • openinfer-higgs-audio/Cargo.toml with higgs-audio feature gate
  • src/lib.rs module declarations
  • Workspace registration in root Cargo.toml

395.2 — Config parsing

  • src/config.rs: HiggsConfig + nested TextConfig + AudioEncoderConfig
  • Nested rope_parameters.rope_theta = 1_000_000 resolution
  • Unit tests assert 11 architecture facts against real checkpoint config

395.3 — Weight name mapping

  • src/weights.rs: map_backbone() maps body.* names to BackboneSlot
  • Audio/codec tensors correctly return None
  • Unit tests verify 398 backbone tensors, all 36 layers with 11 components

395.4 — Backbone forward

  • src/backbone.rs (behind #[cfg(feature = "higgs-audio")])
  • Mirrors openinfer-qwen3-4b weight loading with body.* naming
  • Full 36-layer Qwen3 text prefill: RMSNorm → fused QKV → FlashInfer paged attention → SwiGLU MLP
  • forward(), last_token_logits(), compute_all_position_logits()

395.5 — Backbone parity gate

  • tests/backbone_parity.rs (required-features = ["higgs-audio"])
  • Top-64 logprobs comparison: regret + mean (≤ 0.06 nat) + p99 (≤ 0.20 nat)
  • Same pattern as qwen3 golden gate
  • Clean skip when model or golden file absent

Verification

Mac (completed):

cargo build -p openinfer-higgs-audio    ✅
cargo test --lib (3/3)                  ✅
cargo fmt --all --check                 ✅
git diff main --stat -- <shared crates>  ✅ (empty)

GPU (pending):

  • cargo build -p openinfer-higgs-audio --features higgs-audio
  • Convert backbone_golden.ptbackbone_golden.safetensors
  • cargo test --release -p openinfer-higgs-audio --features higgs-audio --test backbone_parity

Guardrails

  • No shared crate modifications (openinfer-core, openinfer-kernels, etc.)
  • No mod.rs (flat module layout)
  • No unwrap() on external input
  • No weight transposition; no f32→bf16 lossy cast on float tensors

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions