Add a parameter to use LLM as a Judge Descriptor with privately hosted endpoint

[syadavlinklaters]

Description

Hi team,

I am exploring Evidently AI for LLM Evaluation and I came across custom LLM as a Judge Descriptor in which I am particularly interested in. The current API only allows openai models using openai or litellm wrapper. Could you please add an additional parameter wherein we can pass the endpoint of our privately hosted LLMs?

I have attached the API screenshot for your reference.

Thanks and regards,
Satendra

[syadavlinklaters]

https://github.com/evidentlyai/evidently/blob/main/examples/how_to_questions/how_to_use_llm_judge_template.ipynb

[elenasamuylova]

Hi @syadavlinklaters! This functionality is not yet available, but it's definitely on the roadmap.