Make sure to remove previous caches #35
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks @KeremTurgutlu . Does caching still work correctly following this change? |
jph00
reviewed
Oct 18, 2025
| elif isinstance(content, list): c = [_mk_content(o) for o in content] | ||
| else: c = content | ||
| msg = {"role": role, "content": c} | ||
| return _add_cache_control(msg, ttl=ttl) if cache else msg |
There was a problem hiding this comment.
I'm adding this back, since otherwise it's a breaking change to the public API, which AFAICT we don't need to do here. Lemme know if I'm missing something however.
There was a problem hiding this comment.
Good idea! I've only removed it since mk_msg was only used inside mk_msgs and the fact that we use caching by adding cache control to the last 2 messages. So I thought we don't need to add cache control to every message if we are going to remove them anyway, but didn't think of potentially different public API usage.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes sure cache control is removed from all the messages in
mk_msgs, which fixes issues such as #34 when AI makes multiple tool calls > 5.With this fix AI can make 9 tool calls without any issues.
Fixes: #34