Budget cumulative Gemini and Anthropic request history - #185
Open
Anteriousis wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bound cumulative Gemini and Anthropic requests before generation. The default serialized input limit is 512 KiB; above 75%, older bulky successful tool observations become explicit references with original call IDs, sizes and hashes. The latest two tool batches and current live state stay available.
Original user instructions, tool arguments and thought/signature blocks remain intact. Exact source/API reads, failures, pending operations and critical revision/state fields are protected. If protected content still exceeds the limit, return an actionable stop before generation and preserve completed CAD work. This adds no summarization API calls.
README documents context options for the byte budget, optional model context window and Gemini output reserve. Diagnostics distinguish byte-based token estimates from provider-reported usage. Anthropic retains its existing usage reporting; Gemini now records usage if its stream supplies it.
Verification
With local test dependencies on PYTHONPATH, $python denotes the bundled Python executable:
Synthetic comparison: 12 distinct successful calls returning 20,000-character payloads; configured 100,000-byte budget versus disabled budget. Both versions make exactly 13 model requests.
About 55% fewer cumulative serialized bytes in this fixture, without additional model requests. These are mocked byte measurements, not billed-token savings. No paid API calls or full FreeCAD GUI/C++ build were run.
Coverage includes valid call/result pairing, thought signatures, live revisions, failures, pending jobs, source/API reads, image/system/tool payload accounting, output reserve, configuration surviving context refreshes, and stopping before generation for irreducible oversized input.
Compatibility
Limits: serialized bytes divided by four are an estimate, especially for images, not a tokenizer or a guaranteed context-window fit. Unknown model context limits are not guessed. The optional context-window check requires an integrator-supplied limit. Protected content can cause an early stop; omitted observations must be re-read when needed.
Issues
Closes #179.
Before and After Images
Not applicable; no GUI layout changes.