You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What problem or use case are you trying to solve?
The current model info lookup using model_name gets confused with litellm_proxy prefixed models like litellm_proxy/bedrock/us.anthropic.claude-3-5-sonnet-20241022-v2:0, which leads to incorrect model defaults and no prompt caching. There's a need to map this to the correct model name to load the proper 'Model info'.
Describe the UX of the solution the user wants
The user wants a straightforward way to look up model information without manually stripping prefixes or configuring model names.
User's thoughts on the technical implementation?
User noticed the lookup occurs in llm.py. A potential solution involves modifying to accommodate the litellm_proxy prefix, possibly by removing it before processing, as seen in user-specific code changes.
Describe the alternatives considered
Current workaround involves modifying llm.py to strip the prefix manually, but this isn't ideal for all scenarios.
What problem or use case are you trying to solve?
The current model info lookup using
model_name
gets confused withlitellm_proxy
prefixed models likelitellm_proxy/bedrock/us.anthropic.claude-3-5-sonnet-20241022-v2:0
, which leads to incorrect model defaults and no prompt caching. There's a need to map this to the correct model name to load the proper 'Model info'.Describe the UX of the solution the user wants
The user wants a straightforward way to look up model information without manually stripping prefixes or configuring model names.
User's thoughts on the technical implementation?
User noticed the lookup occurs in
llm.py
. A potential solution involves modifying to accommodate thelitellm_proxy
prefix, possibly by removing it before processing, as seen in user-specific code changes.Describe the alternatives considered
Current workaround involves modifying
llm.py
to strip the prefix manually, but this isn't ideal for all scenarios.Additional context
Issue Created By: Steve Cosenza on Slack
Link to Slack Thread: Slack Link
The text was updated successfully, but these errors were encountered: