Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AI Token 限流插件使用示例 #1760

Closed
zlvincent67 opened this issue Feb 13, 2025 · 2 comments · Fixed by higress-group/higress-group.github.io#365 · May be fixed by #1767
Closed

AI Token 限流插件使用示例 #1760

zlvincent67 opened this issue Feb 13, 2025 · 2 comments · Fixed by higress-group/higress-group.github.io#365 · May be fixed by #1767
Assignees

Comments

@zlvincent67
Copy link

官方文档中显示需要依赖 AI 统计插件和AI Proxy插件才能使用,有具体的示例吗?
AI Token 限流插件可以直接接入自己部署的大模型服务,而非大模型提供商的吗?有对应的demo吗?

@nisainan
Copy link

1、官方示例:https://higress.cn/docs/latest/plugins/ai/api-consumer/ai-token-ratelimit/?spm=36971b57.35684624.0.0.67a97cdcMiiLpG#%E5%AE%8C%E6%95%B4%E7%A4%BA%E4%BE%8B

2、可以接入自己的大模型,官方最新版console支持加入自定义的模型服务地址

@cr7258
Copy link
Collaborator

cr7258 commented Feb 14, 2025

@zlvincent67 AI Token 限流插件现在可以不依赖 AI 统计插件了,我晚点更新一下文档。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment