Merge Cache and History, Reduce Critical Section - #32
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request refactors the handling of language model (LM) instances and caching throughout the codebase, replacing the use of
Arc<Mutex<LM>>withArc<LM>for most operations. It also updates the LM construction and caching logic to be more asynchronous and robust, and modifies example usage to match these changes. These improvements simplify concurrency management, make the LM API easier to use, and ensure more consistent caching and history inspection.LM and Adapter API Refactoring
Arc<Mutex<LM>>withArc<LM>in theAdaptertrait and its implementations, includingChatAdapter, removing unnecessary locking and simplifying usage. (crates/dspy-rs/src/adapter/mod.rs,crates/dspy-rs/src/adapter/chat.rs) [1] [2] [3] [4]Arc<LM>instead ofArc<Mutex<LM>>, reflecting the new LM API. (crates/dspy-rs/src/core/settings.rs)LM Construction and Caching
LMbuilder to be asynchronous (build().await) and to initialize the cache handler as anArc<Mutex<ResponseCache>>only when caching is enabled. Also, updated LM response and history tracking to use the cache rather than a local history vector. (crates/dspy-rs/src/core/lm/mod.rs) [1] [2] [3]DummyLMto match the new caching and history inspection APIs, including asynchronous cache insertion to avoid deadlocks. (crates/dspy-rs/src/core/lm/mod.rs)Example Usage Updates
.build().awaitfor LM construction, and removed unnecessaryMutexusage when passing LMs toconfigure. (crates/dspy-rs/examples/01-simple.rs,03-evaluate-hotpotqa.rs,04-optimize-hotpotqa.rs,05-heterogenous-examples.rs,06-oai-compatible-models-batch.rs,07-inspect-history.rs,08-optimize-mipro.rs,09-gepa-sentiment.rs,10-gepa-llm-judge.rs) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]crates/dspy-rs/examples/07-inspect-history.rs,10-gepa-llm-judge.rs) [1] [2]Miscellaneous Improvements
push_messagemethod toChatfor more flexible message handling. (crates/dspy-rs/src/core/lm/chat.rs)Fixes #31