New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] blazing great work about KV Cache: Mooncake #1884

Open

zhyncs opened this issue Jun 28, 2024 · 4 comments

Assignees

Contributor

zhyncs commented Jun 28, 2024 •

edited

Loading

Motivation

TLDR This system has undergone large-scale deployment and validation in the kimi. It has great reference value.

repo:
https://github.com/kvcache-ai/Mooncake

tech report:
https://github.com/kvcache-ai/Mooncake/blob/main/Mooncake-v1.pdf

zhihu:
https://zhuanlan.zhihu.com/p/705754254
https://zhuanlan.zhihu.com/p/705910725

cc @lzhangzz @grimoire @lvhan028

Related resources

No response

Additional context

No response

Contributor Author

zhyncs commented Jun 30, 2024

ref: https://zhuanlan.zhihu.com/p/706097807

Contributor Author

zhyncs commented Jun 30, 2024

ref: https://zhuanlan.zhihu.com/p/706097807

A detailed explanatory article by @feifeibear

lvhan028 assigned irexyc

Collaborator

lvhan028 commented Jul 1, 2024

@irexyc Let's conduct a deep dive into this great work

Contributor Author

zhyncs commented Jul 1, 2024

Microsoft released a blog post about SplitWise in January 2024 https://www.microsoft.com/en-us/research/blog/splitwise-improves-gpu-usage-by-splitting-llm-inference-phases/ similar with Mooncake

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment