MediaConnector::fetch_video takes a VideoFetchConfig { min_frames, max_frames, sample_fps } per call, but AsyncMultiModalTracker always passes VideoFetchConfig::default():
https://github.com/smg-project/llm-multimodal/blob/main/src/tracker.rs#L132
The tracker's public surface is new(media_connector), push_part(part) and finalize(), so a caller that goes through the tracker has no way to choose the decode settings.
That matters for anyone exposing per-request media options. vLLM's OpenAI frontend has a media_io_kwargs request field whose video keys (num_frames, fps) map onto exactly these fields, and it is part of the Rust frontend parity roadmap (vllm-project/vllm#44280). Today that field cannot be honored, because everything reaching the connector goes through the tracker.
Images are already fine, since ImageFetchConfig { detail } is built from the content part itself. Audio is a separate gap, as fetch_audio takes no config at all.
Would you take a small addition that lets the tracker carry a VideoFetchConfig? Happy to extend it to audio too if you want the same shape there.
MediaConnector::fetch_videotakes aVideoFetchConfig { min_frames, max_frames, sample_fps }per call, butAsyncMultiModalTrackeralways passesVideoFetchConfig::default():https://github.com/smg-project/llm-multimodal/blob/main/src/tracker.rs#L132
The tracker's public surface is
new(media_connector),push_part(part)andfinalize(), so a caller that goes through the tracker has no way to choose the decode settings.That matters for anyone exposing per-request media options. vLLM's OpenAI frontend has a
media_io_kwargsrequest field whose video keys (num_frames,fps) map onto exactly these fields, and it is part of the Rust frontend parity roadmap (vllm-project/vllm#44280). Today that field cannot be honored, because everything reaching the connector goes through the tracker.Images are already fine, since
ImageFetchConfig { detail }is built from the content part itself. Audio is a separate gap, asfetch_audiotakes no config at all.Would you take a small addition that lets the tracker carry a
VideoFetchConfig? Happy to extend it to audio too if you want the same shape there.