diff --git a/docs/api-reference/model.mdx b/docs/api-reference/model.mdx index 495c8dd..0a3bb27 100644 --- a/docs/api-reference/model.mdx +++ b/docs/api-reference/model.mdx @@ -162,3 +162,44 @@ results = experiment.run( best_result = experiment.experiment_result.best_run_result ``` + + + +### Overview + +The `VLLMModel` class is a subclass of `Model` that is implemented to work with models served through vLLM. It handles the setup and execution, allowing you to easily integrate vLLM-served models them with your applications. + +### Class Variables + +| Parameter | Type | Default | Description | Required | +|----------------------|----------------------------|--------------------------------|----------------------------------------------------------------------------------------------|----------| +| `api_url` | `Optional[str]` | N/A | URL of model being served through vLLM. | No | +| `required_api_keys` | `ClassVar[Set[str]]` | `{}` | The set of expected API keys for model being served through vLLM. | No | +| `hyperparameters` | `ClassVar[Dict]` | A default hyperparameter search space with `temperature`, `max_tokens`, `top_p`. | A dictionary defining the hyperparameters that can be tuned for the model being served on vLLM. | No | + + +### Example Usage + +```python +from nomadic.model import VLLMModel +from nomadic.tuner import tune + +experiment = Experiment( + model=VLLMModel(api_url="http://localhost:8000/generate"), + params={"temperature", "max_tokens"}, + evaluation_dataset=json.loads( + requests.get( + "https://example.com/evaluation_dataset.json" + ).content + ), + evaluator=SemanticSimilarityEvaluator(embed_model=OpenAIEmbedding()), +) + +results = experiment.run( + param_dict={ + "temperature": tune.choice([0.1, 0.5, 0.9]), + "max_tokens": tune.choice([50, 100, 200]), + }) +best_result = experiment.experiment_result.best_run_result +``` +