[Feature Request] Support llama.cpp for self-hosted models

[lightaime]

Motivation

Support llama.cpp for self-hosted models

Solution

No response

Alternatives

No response

Additional context

No response

[bittoby]

@lightaime @Pakchoioioi Could you provide more detail about it?

Currently llama.cpp models work through Ollama, LM Studio, and OpenAI-compatible endpoints - all three are fully supported in the UI. what exactly should need?

[lightaime]

The original idea is to integrate llama-serve more natively with supports like checking health, getting available models and so on: https://github.com/ggml-org/llama.cpp/tree/master/tools/server if that makes senses @bittoby thanks

[it-education-md]

hi, @lightaime @Pakchoioioi @Wendong-Fan
I'd like to take this one. I've already started and attached a screenshot of the current progress:

please assign it to me. I will open a PR.

[waleedalzarooni]

Assigned, good luck @it-education-md!