-
Notifications
You must be signed in to change notification settings - Fork 380
Open
Description
Hi. I'm working on a proof of concept to run fastagent on Cloudflare workers with Workers AI. This uses a Worker binding to invoke the models instead of a url:
env.AI.run("@cf/meta/llama-3.1-8b-instruct", { "messages": [...] });I'm curious if there is a straightforward way to create a custom model provider to handle this use case. I saw there is a "generic" model API, but it probably wouldn't work for me because it appears to be a singleton. Maybe the new-api in the examples folder would fit my use-case, but I don't see a simple way of extending FastAgentLLM to add a new model definition -- is there a guide for extending it?
Is there anything I'm missing? What are my options?
Metadata
Metadata
Assignees
Labels
No labels