Question about tokenizer_manager.py line 200 : Regarding the preprocessing of multimodal models

[CSEEduanyu]

sglang/python/sglang/srt/managers/tokenizer_manager.py

Line 200 in 6fcd6d7

input_ids = image_inputs["input_ids"]

[CSEEduanyu]

Question 1: According to the code here, the multi-modal inputs_ids is generated in the preprocessing stage, and the process is asynchronous. So the infer of the visual model and the infer of the LLM may happen at the same time. Will there be any conflict?

[CSEEduanyu]

Question 2: In the few multimodal models that have been implemented so far, process_images_async() does not return the input_ids field, so I am confused by the code on line 200 here.

[CSEEduanyu]

My current dilemma is that I want to add support for a new multimodal model, but I don't know whether process_images_async() should return the input_ids of the image mappings or the raw pixel information?