Add optional init kwargs to infer_model()

[Finndersen]

The infer_model() factory function would be much more useful if it allowed providing extra kwargs to also provide to the model for initialisation.

For example, just about all model classes take api_key and http_client as initialisation kwargs, and would be convenient to be able to dynamically initialise a model class while specifying these parameters as well.

[zby]

I would also like to see something like that or maybe better using dependency injection to inject a configured llm client into an Agent - but I don't know how to do that with the current API.

The llm client is usually a field in the AgentModel class (for OpenAI and Anthropic - Gemini uses directly a httpx client) and there is the Model class in between the Agent and the AgentModel.

[Finndersen]

@zby by LLM client do you mean a Model subclass instance? This is already possible, Agent can be initialised with a Model instance

[zby]

No - I mean the llm client object - for example for OpenAI:

client = openai.OpenAI(
    http_client=httpx.Client(
...

this is the object that is initialized with the http_client after all.
Of course it will require some architecture changes to make it work in a typed interface - but Dependency Injection is quite powerful. Passing the http_client to the llm client is a great example of that - you could invent all kinds of additional parameters for the client - like the timeouts, proxy settings - etc. - http_client object encapsulates them all - this is a great abstraction to use.

[Finndersen]

You can provide a http_client when initialising a Model class, and then provide that to the Agent

[zby]

Ah - yeah - indeed there are fields in the OpenAIModel class for both the openai client and the http client (https://github.com/pydantic/pydantic-ai/blob/main/pydantic_ai_slim/pydantic_ai/models/openai.py#L73) - in that case I would not use infer_model for this.

[Finndersen]

Well you could, if this proposed capability was added 😀

[sydney-runkle]

@Finndersen,

Could you provide an API in line with what you're suggesting? I'm not sure I understand the request. Thanks!

[Finndersen]

def infer_model(model: Model | KnownModelName, **init_kwargs) -> Model:
    """Infer the model from the name."""
    if isinstance(model, Model):
        return model
    elif model == 'test':
        from .test import TestModel

        return TestModel()
    elif model.startswith('openai:'):
        from .openai import OpenAIModel

        return OpenAIModel(model[7:], **init_kwargs)
    elif model.startswith(('gpt', 'o1')):
        from .openai import OpenAIModel

        return OpenAIModel(model, **init_kwargs)
    elif model.startswith('google-gla'):
        from .gemini import GeminiModel

        return GeminiModel(model[11:], **init_kwargs)  # pyright: ignore[reportArgumentType]
     ...

model = infer_model("openai:gpt-4o", api_key="my_api_key", http_client=MyHttpClient())

Could perhaps limit the kwargs to just api_key and http_client if they are the only ones that are mostly common across all models.

[sydney-runkle]

I see.

Hmm, infer_model is really just for internal use at the moment, and that you can achieve the same thing by doing Agent(ModelConstructor(model_name, **kwargs)), I don't think this is something we'll pursue.

I think it'd be best to keep infer_model as simple as possible.

[Finndersen]

Where is ModelConstructor?

[sydney-runkle]

Sorry, that's just generic. I meant something like:

Agent(OpenAIModel('gpt-4o', <your kwargs>))

[Finndersen]

Ahh I see. Yes but the point is that dynamic model-selection and initialisation logic in infer_model() is quite useful and probably something that anyone developing agents with configurable model providers (quite common I imagine) would like to use, so IMO would be worth exposing publicly.

But currently it's limited so I have to basically re-implement the whole thing myself just to add that basic extra capability for example.

[VariableExp0rt]

Can this be revisited? I am trying to do the same thing here as @Finndersen with dynamic model selection and this function is rather useful, aside from not being able to pass the client through.