-
Notifications
You must be signed in to change notification settings - Fork 71
[Feature]: Upgrade Plan for UCM Integration with SGLang #872
Copy link
Copy link
Open
Labels
Description
🚀 The feature, motivation and pitch
Motivation
The current UCM integration with SGLang still depends on a version-specific patch originally built for SGLang 0.5.5 (sglang-adapt.patch). This makes it hard to keep up with the latest main branch and adds unnecessary friction for users.
We want to move to a patch-free integration model for SGLang:
- use UCM as a HiCache L3 storage backend on the latest SGLang
main - avoid modifying any SGLang source code
- make the integration available by installing UCM only
Ongoing Work
Todo PR:
PR1 (code): #886
- Add a UCM-side
UnifiedCacheStoreimplementation for SGLang HiCache L3 - Use the SGLang
dynamicbackend to registerunifiedcache - Make the integration auto-activate after installing UCM
PR2 (docs): #891
- Update the SGLang quickstart to describe the new patch-free workflow
- Add new launch examples
- Update the SGLang Dockerfile
Alternatives
No response
Additional context
No response
Reactions are currently unavailable