feat(libsy): affinity - sub-agent aware routing - #125
Conversation
WalkthroughAdds core classifier, processor, and state abstractions; introduces fall-through, random, and affinity routing algorithms; strengthens driver lifecycle and timeout handling; and exposes and documents the new APIs. ChangesComposable routing
Estimated code review effort: 5 (Critical) | ~120 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
crates/libsy/src/algorithms/fall_through.rs (1)
92-139: 🚀 Performance & Scalability | 🔵 Trivial | ⚖️ Poor tradeoffOptional: release the session lock before the routed LLM call.
stateis not read or mutated after the decision replay (Line 134), yet the guard is held acrosscall_llm_target(Lines 136-139). For a shared-session router this serializes concurrent turns for the entire duration of the model call, not just the fact-fold. Dropping the guard after step 4 would keep the fold atomic while shortening contention. If holding it across the call is intentional (e.g. strict per-session ordering), disregard.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/libsy/src/algorithms/fall_through.rs` around lines 92 - 139, The session state lock in the fall-through routing method remains held during call_llm_target even though state is no longer used after decision replay. Release the state guard immediately after the processor loop handling Event::Decision, before invoking driver.call_llm_target, while preserving atomicity through the fact-fold and decision replay.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/libsy/src/algorithms/affinity.rs`:
- Around line 31-33: Update the comment for MAX_ASSIGNMENTS to describe eviction
as removing an arbitrary entry when the bound is reached, matching evict_if_full
and HashMap iteration semantics; remove the claim that the oldest entry is
evicted.
In `@crates/libsy/src/core/driver.rs`:
- Around line 146-157: Add a concise Rust doc comment to the private
ensure_started method describing that it verifies stream() was called before
publish operations proceed, and preserves its existing error behavior.
- Around line 67-68: Update FULFILL_REQUEST_TIMEOUT to represent the intended
10-minute default by changing its duration value accordingly, while preserving
the existing timeout usage and configurability TODO.
---
Nitpick comments:
In `@crates/libsy/src/algorithms/fall_through.rs`:
- Around line 92-139: The session state lock in the fall-through routing method
remains held during call_llm_target even though state is no longer used after
decision replay. Release the state guard immediately after the processor loop
handling Event::Decision, before invoking driver.call_llm_target, while
preserving atomicity through the fact-fold and decision replay.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 517c5aa7-fce6-44af-94cc-086f286ef99d
📒 Files selected for processing (12)
crates/libsy/README.mdcrates/libsy/src/algorithms.rscrates/libsy/src/algorithms/affinity.rscrates/libsy/src/algorithms/fall_through.rscrates/libsy/src/algorithms/random_classifier.rscrates/libsy/src/core.rscrates/libsy/src/core/algorithm.rscrates/libsy/src/core/classifier.rscrates/libsy/src/core/driver.rscrates/libsy/src/core/processor.rscrates/libsy/src/core/state.rscrates/libsy/src/lib.rs
Signed-off-by: ayushag <ayushag@nvidia.com>
6ae1be6 to
d7ebe50
Compare
Signed-off-by: Greg Clark <grclark@nvidia.com>
e7e766c to
f102bb6
Compare
What
Short summary of what this PR changes.
Why
The motivation — what problem does this solve, or which ticket does it close?
Closes #
How tested
uv run ruff check .cleanuv run mypy switchyardcleanuv run pytest tests/greenChecklist
snake_caseof the primary class.switchyard/__init__.py.__all__if intended for downstream use.--helpupdated if customer-facing surface changed.Signed-off-by: Your Name <email>) per the DCO.Notes for reviewers
Anything reviewers should pay extra attention to — risky paths, follow-up tickets, intentional trade-offs.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation