Add Cohere v2/chat API support #15722
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| if extra_headers is not None: | ||
| headers.update(extra_headers) | ||
|
|
||
| verbose_logger.debug(f"Model: {model}, API Base: {api_base}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
The correct fix is to ensure that sensitive values such as api_base and model are not logged if they may contain or be derived from user-provided secrets such as API keys/tokens. We should:
- Scrub/obfuscate potentially sensitive data before logging, or
- Avoid logging these fields altogether unless we're confident (through sanitization) that they cannot contain secrets.
Best approach:
- Implement a helper function (
sanitize_urlor similar) for logging which strips query strings, authentication info, and sensitive path segments fromapi_baseandmodelbefore logging. - Update the verbose logger call on line 2461 to use sanitized versions.
- This ensures no sensitive data is leaked to logs while preserving helpful debug information for troubleshooting.
Edits needed:
- Define a
sanitize_for_loggingfunction inlitellm/main.py, e.g., near the imports or top-of-file utility region. - Use
sanitize_for_logging(model)andsanitize_for_logging(api_base)in the debug log at line 2461.
| @@ -41,6 +41,45 @@ | ||
|
|
||
| from litellm._uuid import uuid | ||
|
|
||
|
|
||
|
|
||
| def sanitize_for_logging(val): | ||
| """Remove sensitive data from strings (like URLs or tokens). | ||
| - For URLs, removes query parameters and userinfo. | ||
| - For other str, if looks like an API key, redacts. | ||
| """ | ||
| import re | ||
| from urllib.parse import urlparse, urlunparse | ||
|
|
||
| if not isinstance(val, str): | ||
| return val | ||
|
|
||
| # If the string looks like a URL, strip userinfo and query | ||
| try: | ||
| p = urlparse(val) | ||
| netloc = p.hostname or "" | ||
| if p.port: | ||
| netloc += f":{p.port}" | ||
| sanitized = urlunparse(( | ||
| p.scheme, | ||
| netloc, # omit user:pass@ | ||
| p.path, | ||
| p.params, | ||
| '', # strip query | ||
| '', # strip fragment | ||
| )) | ||
| if p.scheme and p.netloc: | ||
| return sanitized | ||
| except Exception: | ||
| pass | ||
|
|
||
| # Redact likely API keys/tokens (crude: long alphanum strings) | ||
| if re.fullmatch(r"[A-Za-z0-9_\-\.=]{20,}", val): | ||
| return "[REDACTED]" | ||
|
|
||
| # Redact if the string contains 'key=' or 'token=' | ||
| return re.sub(r'((key|token)=)[^&;]+', r'\1[REDACTED]', val, flags=re.IGNORECASE) | ||
|
|
||
| if TYPE_CHECKING: | ||
| from aiohttp import ClientSession | ||
|
|
||
| @@ -2458,7 +2497,7 @@ | ||
| if extra_headers is not None: | ||
| headers.update(extra_headers) | ||
|
|
||
| verbose_logger.debug(f"Model: {model}, API Base: {api_base}") | ||
| verbose_logger.debug(f"Model: {sanitize_for_logging(model)}, API Base: {sanitize_for_logging(api_base)}") | ||
| verbose_logger.debug(f"Provider Config: {provider_config}") | ||
| response = base_llm_http_handler.completion( | ||
| model=model, |
| except Exception as e: | ||
| pytest.fail(f"Error occurred: {e}") | ||
|
|
||
|
|
There was a problem hiding this comment.
can you add a test "documents" and "citation_options" is getting sent in the request body
There was a problem hiding this comment.
Added test_cohere_documents_citation_options_in_request_body
| os.environ["COHERE_API_KEY"] = "cohere key" | ||
|
|
||
| # cohere call | ||
| # cohere v1 call |
There was a problem hiding this comment.
i believe /v2 should be the default @Sameerlite
it's likely /v1 will be deprecated soon, so it's safer to move to v2 at this point (it's been around for quite a while, so i would assume it's stable)
@ishaan-jaff and I discussed this. This might break stuff for people relying on v1 api. So this should be done gradually |
Yes, made v2 default |
Sameerlite
merged commit c74d5c9
into
litellm_sameer_oct_staging
3 participants
Title
Add Cohere v2/chat API support
Relevant issues
Fixes #13311
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🆕 New Feature
Changes
I have used concept getting used for bedrock models, where we detect the API to be used in chat based on the suffix after the llm_provider. I have changed the a file's name to CohereV2ChatPassthroughConfig as it was getting used for passthrough code.
Test