New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docker based or BareMetal serving #205

Open

dhandhalyabhavik opened this issue Nov 10, 2024 · 1 comment

dhandhalyabhavik commented Nov 10, 2024

Wondering if any plans to implement to enable servings,

similar to vllm serving, it should support OpenAI compatible chat endpoints.

gnusupport commented Jan 6, 2025

I would like to get any kind of loaded serving on endpoint, what is the way to serve it on endpoint?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment