refactor(multimodal): move audio-processor selection onto ModelProcessorSpec

[slin1237]

Description

Problem

#1905 left two overlapping model→audio-processor mechanisms: ModelRegistry (family → ModelProcessorSpec) and a separate AudioProcessorRegistry keyed by the spec-name string ("qwen3_asr"/"qwen3_omni"). That string must be registered in two independent places and kept in sync; a mismatch is a runtime error with no compile-time guarantee.

Solution

Move audio-processor selection onto the spec: ModelProcessorSpec::audio_processor(model_config, preprocessor_config) -> Option<Box<dyn AudioPreProcessor>> (default None). The spec that owns a model's prompt/placeholder logic now owns its audio preprocessor, and AudioProcessorRegistry is deleted. Construction is identical (Qwen3AudioProcessor::from_configs(...)), so behavior is preserved. Vision keeps its own VisionProcessorRegistry on purpose — folding it in would change behavior (specs match broadly, the vision registry narrowly).

Changes

• registry/traits.rs: audio_processor trait method (default None).
• registry/{qwen3_asr,qwen3_omni}.rs: override to build Qwen3AudioProcessor.
• audio/processor.rs: delete AudioProcessorRegistry/AudioProcessorFactory (keep the AudioPreProcessor trait).
• multimodal/{config,process}.rs: drop the registry; call spec.audio_processor(...).
• Tests migrated to the spec path + a llava no-audio test.

Test Plan

cargo test -p llm-multimodal and cargo test -p smg --lib routers::grpc::multimodal pass; audio construction is byte-identical to before (registry deletion only changes selection wiring, not preprocessing output).

Checklist

• cargo +nightly fmt passes
• cargo clippy --all-targets --all-features -- -D warnings passes
• (Optional) Documentation updated
• (Optional) Please join us on Slack #sig-smg to discuss, review, and merge PRs

[coderabbitai]

📝 Walkthrough

Walkthrough

Audio processor selection moved from a shared registry into ModelProcessorSpec::audio_processor. Qwen3 ASR and Omni provide implementations, LLaVA verifies the default absence, and gateway preprocessing now uses the resolved model specification directly.

Changes

Audio processor selection

Layer / File(s)	Summary
Processor contract and registry removal crates/multimodal/src/audio/*, crates/multimodal/src/registry/traits.rs, crates/multimodal/src/lib.rs	The shared audio processor registry and its public re-exports were removed, while ModelProcessorSpec gained an optional audio processor hook.
Model-specific audio wiring crates/multimodal/src/registry/qwen3_asr.rs, crates/multimodal/src/registry/qwen3_omni.rs, crates/multimodal/src/registry/llava.rs	Qwen3 ASR and Omni construct audio processors and test preprocessing output shapes; LLaVA tests that no processor is provided.
Gateway preprocessing integration model_gateway/src/routers/grpc/multimodal/{config,process}.rs	Gateway preprocessing passes the model specification object and obtains audio processors through its new hook instead of a component registry.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MultimodalPlan
  participant ModelProcessorSpec
  participant Qwen3AudioProcessor
  MultimodalPlan->>ModelProcessorSpec: pass spec to preprocess_modality
  MultimodalPlan->>ModelProcessorSpec: request audio_processor
  ModelProcessorSpec->>Qwen3AudioProcessor: construct processor
  Qwen3AudioProcessor-->>MultimodalPlan: preprocess audio clips

Loading

Possibly related PRs

• lightseekorg/smg#593: Restructures the multimodal registry and overlaps with the model specification changes.
• lightseekorg/smg#1905: Introduces the audio processor registry and directly overlaps with its replacement.

Suggested labels: tests

Suggested reviewers: key4ng, catherinesue, gongwei-130

Poem

I’m a rabbit hopping through the spec,
Qwen3’s audio gets a clever check.
Old registries fade away,
Model hooks now lead the way,
LLaVA rests with none to fetch! 🐇

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main refactor: moving audio-processor selection onto ModelProcessorSpec.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch refactor/unify-processor-selection

---

_{Comment @coderabbitai help to get the list of available commands.}

[claude]

👋 The PR description doesn't fully follow
PULL_REQUEST_TEMPLATE.md:

• Missing header: ## Changes (found ### Changes — should be h2 not h3)
• Missing header: ## Test Plan (found ### Test Plan — should be h2 not h3)

Please update the PR description so reviewers have the context they need.

[claude]

Clean, well-scoped refactor. No bugs or issues found. The single-source-of-truth pattern (spec owns its audio processor alongside prompt/placeholder logic) is a clear improvement over the string-keyed registry.

[gemini-code-assist]

Code Review

This pull request refactors the audio preprocessor selection logic by removing the centralized AudioProcessorRegistry and instead delegating the creation of audio preprocessors directly to the ModelProcessorSpec trait implementations (such as Qwen3AsrSpec and Qwen3OmniSpec). This simplifies the design by making the model spec the single source of truth for audio processor selection. I have no feedback to provide as there are no review comments.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.