Repository navigation
CACHE TIMER: add a cache timer and auto ping - Avoid CACHE HITS and keep CACHE WARM #38532
lucioaleixofilho
started this conversation in
Ideas
Replies: 1 comment
|
Seconded. I made a |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Add a cache timer on the righ side (at least for Claude - don´t know how usefull it would be for other providers) and and auto-ping on/off button to keep the cache warm every 55 minutes - allow to set the limit the ammount of "turns" this keeps on otherwise it becomes counter effective, not cost efficient. Whenever the software restarts this turns off automatically (unless you can set this at Claude CLI level and make it "live" there)
I have been doing this with automations in T3 Code but apparently I have to set a new one for each session. When I restart the software, it ends of pinging all my sessions and if the cache was expired, I get a big cache hit for inactive sessions.
Before T3, I set this up in Claude Code and it worked pretty well. I don´t know how you do the "Compact idle threads" (how you measure the cache timer - but can be an extension of that.
BTW, the "Compact idle threads" feature does not work if the session has a "pending input" open for the user, so if you do not change the model choice before answering, you will get a big hit.
thanks
All reactions