[Feature] Save cache from requests and load

[SinanAkkoyun]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

This might be difficult to implement, but I am facing the following issue:
When running Qwen2-VL on bigger images, the preprocessor takes a long time to convert the images to tokens.

It would be awesome if we could have a way (OpenAI API with extra parameters) to tell the backend to store the cache of a request and load it by ID for another request, which would make it possible to not reprocess every image (and prompt in general) on each call.

If my problem could be solved in an easier way I would be thankful for any input :)

Related resources

No response