Implement per-user workflow and function instance with lazy initialization #1151
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
…flow-per-mcp-user Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
…flow-per-mcp-user Signed-off-by: Yuchen Zhang <[email protected]>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
…flow-per-mcp-user Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
…ders (NVIDIA#1143) Make it so we only forward what has been specified to LLM and Embedder providers. This gives the flexibility and control to end-users rather than the toolkit being naturally unable to support newer models with added imposed restrictions (such as GPT-5 and Claude Sonnet 4.5) - Remove `TopPMixin` and `TemperatureMixin` due to not being needed - Add `exclude_unset=True` to all LLM and Embedder `model_dump` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **Documentation** * Removed deprecated gated-field mixin examples and simplified gated-fields docs. * **Refactor** * Exposed temperature and top-p as explicit, consistent configurable model parameters across providers. * Standardized client serialization to omit unset configuration fields when building LLM clients. * **Tests** * Removed legacy mixin-specific tests and updated assertions to align with new defaults. * **Examples** * Added explicit temperature settings in several example configs. Authors: - Will Killian (https://github.com/willkill07) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: NVIDIA#1143 Signed-off-by: Yuchen Zhang <[email protected]>
* Applies fix for NVIDIA#1139 from PR NVIDIA#1154 * Moves `src/nat/tool/code_execution/test_code_execution_sandbox.py` -> `tests/nat/tools/test_code_execution_sandbox.py` allowing the tests to be discovered by pytest. * Update tests to also test code execution in a workflow and using piston. * Slim down the local sandbox container to just what is needed (~7GB down to ~500MB) * Add a / endpoint to the local sandbox server allowing a simple health check. Closes NVIDIA#1139 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **New Features** * Added a local sandbox service with a health-check endpoint; sandbox URL exposed via a CI environment variable. * **Improvements** * Sandbox runtime simplified and standardized on a minimal Python 3.13 image with updated startup/configuration and dependencies for more consistent execution. * **Tests** * Replaced the previous sandbox test suite with a comprehensive pytest-driven module covering execution, errors, I/O, timeouts, and workflow paths. * **Chores** * CI pipeline updated to provision and run the local sandbox during tests. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#1162 Signed-off-by: Yuchen Zhang <[email protected]>
* Includes unrelated documentation fix where back-ticks were not being rendered properly in the Sphinx TOC ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit ## Release Notes * **Documentation** * Added a comprehensive Security Considerations guide with practical risk mitigation and best practices. * Added navigation links to the new Security Considerations resource. * Minor tutorial formatting fix. * **Security / Policy** * Removed the previous standalone security reporting guidance and contact instructions. * **Chores** * Expanded accepted security-related vocabulary and minor path-filtering allowance. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#1164 Signed-off-by: Yuchen Zhang <[email protected]>
Closes - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. * **Documentation** * Updated local documentation preview instructions to use a static server for improved accessibility. * Added clarification on version switcher behavior when accessing documentation locally. * Updated documentation version URLs to absolute NVIDIA documentation links. Authors: - Will Killian (https://github.com/willkill07) - Patrick Chin (https://github.com/thepatrickchin) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: NVIDIA#1167 Signed-off-by: Yuchen Zhang <[email protected]>
This PR enables functions associated with custom routes to accept arbitrary JSON payloads of any type (objects, arrays, strings, numbers, booleans). Previously, custom route function parameters defined a fixed request body schema, making them incompatible with arbitrary JSON structures. This enhancement allows custom routes to accept any valid JSON request body, regardless of its schema. ```bash curl -X POST http://localhost:8000/get_request_metadata \ -H 'Content-Type: application/json' \ -d '[1, 2, 3, 4, 5]' ``` ```bash curl -X POST http://localhost:8000/get_request_metadata -d '"hello world"' curl -X POST http://localhost:8000/get_request_metadata -d '42' curl -X POST http://localhost:8000/get_request_metadata -d 'true' ``` None. This is a backward-compatible enhancement. Existing custom routes continue to work, and new routes can opt into arbitrary JSON support by using `RootModel[typing.Any]`. - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. * **Documentation** * Expanded "Simple Calculator - Custom Routes and Metadata Access" with reorganized TOC, new sections on defining custom routes and metadata access, updated curl examples, renamed response heading, and added request-body examples (JSON array/string/number/boolean). Notes on accepting arbitrary JSON payloads and limitations for certain custom routes. * **Chores** * Route registration adjusted to skip async-generation routes for custom endpoints; request handling and examples updated to include request-body data alongside other metadata in responses. Authors: - Eric Evans II (https://github.com/ericevans-nv) - David Gardner (https://github.com/dagardner-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#1163 Signed-off-by: Yuchen Zhang <[email protected]>
Closes - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. - Documentation - Added release notes for v1.3.1 with an Overview, categorized changes, and changelog reference; updated Quick Start UI docs, installation reminders, reasoning diagrams, and SECURITY notes. - New Features - Added support for the Claude Sonnet 4.5 model and support for arbitrary JSON body types in custom routes. - Bug Fixes - Fixed issues including config serialization, multi-user demo sessions, tool parameter handling, Async Chat reliability, and code-execution tool behavior. - Chores - Updated changelog and acknowledged a new contributor. Authors: - Will Killian (https://github.com/willkill07) - David Gardner (https://github.com/dagardner-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: NVIDIA#1166 Signed-off-by: Yuchen Zhang <[email protected]>
* Reverts PR NVIDIA#1164 * Existing `SECURITY.md` file is retained * New document `docs/source/resources/security-considerations.md` is retained ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **Documentation** * Added security documentation with vulnerability reporting procedures, contact information, and security disclosure guidance. * Updated documentation references. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#1168 Signed-off-by: Yuchen Zhang <[email protected]>
Closes ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **Improvements** - Extended valid temperature range for LLM model configurations across multiple providers (AWS Bedrock, Azure OpenAI, LiteLLM, NIM, and OpenAI), allowing higher values for greater output randomness control. - Updated temperature parameter descriptions for clarity across all affected configurations. Authors: - Will Killian (https://github.com/willkill07) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: NVIDIA#1172 Signed-off-by: Yuchen Zhang <[email protected]>
…1171) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **Tests** * Updated a session-management test to accept injected mocks and use a locally constructed instance instead of a shared fixture. * Adjusted timeout expectation in the test to 2.0s and updated assertions accordingly. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#1171 Signed-off-by: Yuchen Zhang <[email protected]>
…n once (NVIDIA#1175) * Fix the logic in `ADKProfilerHandler` preventing the patch from being applied more than once. * Since the patches set attributes directly on the `ADKProfilerHandler` instance, concurrent `ADKProfilerHandler` instances are not possible, thus `ADKProfilerHandler` is now a singleton. ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **Improvements** * Enhanced profiler handler reliability with improved instance lifecycle management and automatic recovery * Added thread-safe initialization for concurrent operations * Strengthened instrumentation state tracking with improved restoration capabilities * **Tests** * Added comprehensive test coverage for profiler independence and instrumentation cleanup verification Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#1175 Signed-off-by: Yuchen Zhang <[email protected]>
…IA#1177) * Ensures that the 1.0 compatibility loader is removed after performing the compatibility tests. Since the rest of the tests should not depend on it, we should ensure that it is removed so that we don't have other tests silently depending on it. ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **Tests** * Enhanced test suite to automatically restore import-related state and remove compatibility helpers between runs, improving isolation and preventing cross-test interference. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#1177 Signed-off-by: Yuchen Zhang <[email protected]>
…functions (NVIDIA#1130) This PR introduces *Middleware*, a component-based middleware layer for functions in the NeMo Agent Toolkit (NAT). Middleware are first-class components that wrap function execution with preprocessing and postprocessing logic, enabling features like caching, logging, authentication, rate limiting, validation, and retry behavior — without modifying function code. Like middleware in Express.js or FastAPI, middleware support composable, reusable logic applied across many functions consistently. Agent workflows often require consistent cross-cutting concerns such as: - Caching - Logging and metrics - Authentication and access control - Rate limiting - Input validation - Retry logic - Request/response transformation Middleware remove code duplication and tightly coupled logic by making these concerns configurable and reusable. Middleware follow existing NAT component patterns: - Defined in YAML under `middleware` - Built by the workflow builder - Referenced by name using `middleware` in every `Function`'s config - Resolved before function construction (dependency ordering) - Registered via global component type registry ```yaml middleware: api_cache: _type: cache enabled_mode: always similarity_threshold: 1.0 request_logger: _type: logging_middleware log_level: INFO include_inputs: true functions: my_api_function: _type: api_caller endpoint: https://api.example.com middleware: ["request_logger", "api_cache"] ``` ```python @register_middleware(config_type=CacheMiddlewareConfig) async def cache_middleware(config, builder): yield CacheMiddleware(...) ... ``` Execution follows a four-phase pattern: 1. Preprocess (inspect or modify inputs) 2. Call next (propagate execution — next middleware or function) 3. Postprocess (inspect or modify output) 4. Continue (return final result) Chained middleware wrap the function in a layered "onion" pattern: ``` Request → Logger(pre) → Validator(pre) → Cache(pre) → Function ↓ Response ← Logger(post) ← Validator(post) ← Cache(post) ← ``` ```python class LoggingMiddlewareConfig(FunctionMiddlewareBaseConfig, name="logging_middleware"): log_level: str = "INFO" include_inputs: bool = True include_outputs: bool = True ``` ```python class LoggingMiddleware(FunctionMiddleware): async def function_middleware_invoke(self, value, call_next, context): ... ``` ```python @register_middleware(config_type=LoggingMiddlewareConfig) async def logging_middleware(config, builder): yield LoggingMiddleware(...) ``` Supports: * Exact or fuzzy matching * Eval-mode-only caching * Streaming-safe behavior * Fallback on serialization failures New core components: * `function_middleware.py` — base class, chaining, context * `cache_middleware.py` — built-in caching layer * `data_models/middleware.py` — config base class * Builder + registry integration updates * 400+ new lines of tests for middleware architecture * YAML parsing + builder integration coverage * Streaming and error handling tests * Full pass across existing test suite (no required changes) A new reference page has been added: * `docs/source/reference/middleware.md` Includes design overview, examples, testing guidance, best practices, and migration help. Authors: - Dhruv Nandakumar (https://github.com/dnandakumar-nv) - Claude (https://github.com/claude) Approvers: - Eric Evans II (https://github.com/ericevans-nv) - Will Killian (https://github.com/willkill07) URL: NVIDIA#1130 Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
…flow-per-mcp-user Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
…flow-per-mcp-user Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
…flow-per-mcp-user Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Signed-off-by: Yuchen Zhang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Closes
By Submitting this PR I confirm: