Skip to content

[Feature Request] Allow use of non OAI-like API url #9

@fgbelidji

Description

@fgbelidji

Currently the benchmarker can only work with an endpoint accessible through an OAI-like API url set in --url. It would be great to extend this to any type of endpoint (e.g Sagemaker endpoint) and allow to set up a LLM client that can receive requests from the load tester.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions