Skip to content

[Feature]: Upgrade Plan for UCM Integration with SGLang #872

@pyxyzc

Description

@pyxyzc

🚀 The feature, motivation and pitch

Motivation

The current UCM integration with SGLang still depends on a version-specific patch originally built for SGLang 0.5.5 (sglang-adapt.patch). This makes it hard to keep up with the latest main branch and adds unnecessary friction for users.

We want to move to a patch-free integration model for SGLang:

  • use UCM as a HiCache L3 storage backend on the latest SGLang main
  • avoid modifying any SGLang source code
  • make the integration available by installing UCM only

Ongoing Work

Todo PR:

PR1 (code): #886

  • Add a UCM-side UnifiedCacheStore implementation for SGLang HiCache L3
  • Use the SGLang dynamic backend to register unifiedcache
  • Make the integration auto-activate after installing UCM

PR2 (docs): #891

  • Update the SGLang quickstart to describe the new patch-free workflow
  • Add new launch examples
  • Update the SGLang Dockerfile

Alternatives

No response

Additional context

No response

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions