chore: merge release/0.5 into main (#1829)

* feat(studio): add trace graph view (#1791)

* feat(studio): add trace graph view

Signed-off-by: Yamini <ykagal@gmail.com>

* fixes for default off FF, overflow, code split

Signed-off-by: Sean Teramae <steramae@nvidia.com>

* add record to e2e test

Signed-off-by: Sean Teramae <steramae@nvidia.com>

* more minor fixes

Signed-off-by: Sean Teramae <steramae@nvidia.com>

---------

Signed-off-by: Yamini <ykagal@gmail.com>
Signed-off-by: Sean Teramae <steramae@nvidia.com>
Co-authored-by: Sean Teramae <steramae@nvidia.com>

* fix(files): block fileset delete when models or adapters still reference it (#1761)

* fix(files): block fileset delete when models or adapters still reference it

A fileset used as model or adapter weights could be deleted with no check, leaving a dangling entity and a late 404 during job download. Fail closed with 409 when dependents exist, 503 when Entity Store cannot be queried, and validate the weights fileset at customization submit.

Signed-off-by: Tyler Bray <tbray@nvidia.com>

* fix(files): sanitize fileset delete-guard logs and document 409/503 detail

CodeQL flagged unsanitized workspace/name on the new 503 path. Document the FastAPI string-detail envelope on DELETE so OpenAPI matches the actual 409 and 503 bodies.

Signed-off-by: Tyler Bray <tbray@nvidia.com>

* chore(sdk): sync Python SDK with fileset DELETE 409 and 503

Stainless regeneration picks up the documented error envelope so lint-python-sdk matches the OpenAPI spec.

Signed-off-by: Tyler Bray <tbray@nvidia.com>

* fix(files,entities): map fileset ref lookup outages to 503 and delete models before filesets

EntityClient.list raises NemoClientError, not EntityStoreError, so the documented 503 path never fired on a real Entities outage. Workspace cleanup now removes adapters and models before filesets so same-workspace teardown is not blocked by the 409 guard.

Signed-off-by: Tyler Bray <tbray@nvidia.com>

---------

Signed-off-by: Tyler Bray <tbray@nvidia.com>

* docs(studio): add Custom Models page for the Studio customization UI (#1817)

* docs(studio): add Custom Models page for the Studio customization UI

Documents the Fine-tune a Model form (backend, model, training method,
dataset, parameters, LoRA/DPO, compute), the Custom Models list, job
cloning, and the job details page, matching release/0.5 where the
customize-method modal and Prompt-tuned option have been removed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Henrique Tolentino <htolentino@nvidia.com>

* docs(studio): document GRPO training-health panels and per-backend training method fields

Cover the Overview tab's GRPO reward/training-health panels (vs. the
training-loss panel used by SFT/distillation/DPO), and correct the
Training Method section description to reflect the actual per-backend
fields (fine-tuning type + SFT/Distillation for Automodel/Unsloth,
DPO/GRPO for RL).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Henrique Tolentino <htolentino@nvidia.com>

---------

Signed-off-by: Henrique Tolentino <htolentino@nvidia.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* feat(skills): Switch build agent skill to use Fabric and LC DeepAgents (#1779)

* Refine build agent skill for Fabric

Clarify the supported Deep Agents and Fabric workflow, add non-happy-path guidance, and package local evaluation assets.

Signed-off-by: Yamini <ykagal@gmail.com>

* Harden build agent onboarding and dependency boundary

Explain Ethos and Fabric before implementation, keep Deep Agents dependencies
inside the generated customer project, and cover the boundary in skill tests.

Signed-off-by: Yamini <ykagal@gmail.com>

* Align build agent harness dependency checks

Teach the build agent skill to detect and reuse the Agents plugin harness,
ask before installing the optional plugin, and stop on partial installs.
Document source checkout packaging and add dependency failure evals.

Signed-off-by: Yamini <ykagal@gmail.com>

* Update build agent benchmark evidence

Record the nine current evaluation tasks and the successful packaged Fabric
trajectory. Use the repository standard NVSkills-Eval terminology and keep
formal Tier 3 results marked as pending.

Signed-off-by: Yamini <ykagal@gmail.com>

* Remove build agent review artifacts from package

Keep behavioral evaluations in source while removing the generated gate marker,
hand written benchmark and runtime packaging rule for evaluation data.

Signed-off-by: Yamini <ykagal@gmail.com>

* fix(skills): add copyright headers

Signed-off-by: Yamini <ykagal@gmail.com>

* fix(skills): defer agent registration until gates pass

Signed-off-by: Manjesh Mogallapalli <mmogallapall@nvidia.com>

* fix(skills): preserve agent deployment scope

Signed-off-by: Manjesh Mogallapalli <mmogallapall@nvidia.com>

* fix(skills): clarify optional agent environment

Signed-off-by: Manjesh Mogallapalli <mmogallapall@nvidia.com>

---------

Signed-off-by: Yamini <ykagal@gmail.com>
Signed-off-by: Manjesh Mogallapalli <mmogallapall@nvidia.com>
Co-authored-by: Manjesh Mogallapalli <mmogallapall@nvidia.com>

* AIRCORE-1121 Update agent build guidance for Fabric (#1780)

Point new agent builds to the Fabric based skill while keeping explicit
maintenance and migration work for existing NAT workflows available.

Signed-off-by: Yamini <ykagal@gmail.com>

---------

Signed-off-by: Yamini <ykagal@gmail.com>
Signed-off-by: Sean Teramae <steramae@nvidia.com>
Signed-off-by: Tyler Bray <tbray@nvidia.com>
Signed-off-by: Henrique Tolentino <htolentino@nvidia.com>
Signed-off-by: Manjesh Mogallapalli <mmogallapall@nvidia.com>
Signed-off-by: Swarom Muley <smuley@nvidia.com>
Co-authored-by: Yamini <yamini@users.noreply.github.com>
Co-authored-by: Sean Teramae <steramae@nvidia.com>
Co-authored-by: tylersbray <tbray@nvidia.com>
Co-authored-by: htolentino-nvidia <htolentino@nvidia.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Manjesh Mogallapalli <mmogallapall@nvidia.com>

Author: 7 people

AGENTS.md

@@ -20,7 +20,7 @@ If a user asks you to set up, try, build, evaluate, harden, or optimize an agent
 
 - DO NOT invoke any plugin-based skill, `/skill-name` slash command, or globally-installed assistant for these requests. The repo skills are authoritative. A globally-installed skill (brainstorming, planning, code-review, etc.) will give the wrong answer because it does not know NeMo.
 - DO NOT brainstorm a solution from scratch when a NeMo skill claims the task. The skill already has the answer.
-- DO NOT write Python, Pydantic AI, LangChain, or any agent framework code from scratch. NeMo Platform uses the NVIDIA NeMo Agent Toolkit (NAT) under the hood. Skills will tell you the right way to wire your agent.
+- DO NOT write Python, Pydantic AI, LangChain, or other agent framework code from scratch when a shipped skill covers the request. Skills will tell you the supported way to connect the agent to NeMo Platform. For a new agent build, `nemo-build-agent` uses LangChain Deep Agents through Fabric. Work on a NAT workflow only when the user explicitly asks to maintain or migrate an existing one.
 - DO NOT improvise CLI flags. Only use flags documented in the skill or shown in `nemo <subcommand> --help`.
 - DO NOT report a task complete if you cannot verify it. If a verification step fails or times out, surface what you saw and ask the user to confirm before continuing.
 
@@ -32,7 +32,7 @@ User-facing skills in `packages/nemo_platform_ext/src/nemo_platform_ext/skills/`
 - `setup`: verifies that NeMo Platform is installed and running. If install is missing, tells the user how to run the CLI install (`make bootstrap` + `nemo setup`). **Install itself is CLI-only.** Do not attempt to install NeMo via skill-driven pip; the workspace dependency graph and credential handling are not reliably automatable inside a sandbox.
 - `nemo-explore`: design conversation that feeds into an Ethos. Always confirms purpose, principles, and vision.
 - `nemo-ethos`: writes `agents/<name>-ethos/ETHOS.md` from explore output, then shows a gut-check of the agent.
-- `nemo-build-agent`: scaffolds NAT workflow YAML from the Ethos and deploys.
+- `nemo-build-agent`: builds a tested LangChain Deep Agent from an approved Ethos, then packages and registers it through Fabric.
 - `nemo-try-agent`: test a deployed agent or chat with a model.
 - `nemo-intake`: instrument agents, choose an ingest format, upload/query telemetry, and attach evaluator results.
 - `nemo-experiments-upload`: publish named evaluation runs and scores to the Experiments leaderboard.
@@ -56,9 +56,9 @@ NeMo Platform brings together NVIDIA NeMo libraries under one CLI, Python SDK, a
 - **Harden agents**: guardrails (content safety, jailbreak detection, PII redaction), auditor (red-teaming via garak), anonymizer (PII handling for training data).
 - **Evaluate agents**: evaluator (LLM-as-judge, deterministic, agentic, RAG benchmarks), Harbor-backed eval suites.
 - **Tune agents and models**: skill optimization, prompt/hyperparameter tuning, Switchyard model routing, and fine-tuning through Customizer.
-- **Build agents**: NeMo Agent Toolkit (NAT) for LangGraph-based agents. Broader framework support on the roadmap.
+- **Build and manage agents**: Fabric connects supported agent harnesses to NeMo Platform for packaging, deployment, testing, observation, and optimization.
 
-NeMo Platform optimizes LangGraph agents wrapped in NAT today. Other frameworks require a user-written NAT wrapper. Be honest about this when users ask.
+New agents should use an explicit Fabric configuration and a supported adapter. Existing NAT workflows can still be maintained when the user asks for that path. Do not create a NAT wrapper as the default way to bring a new agent into NeMo Platform.
 
 ---
 

docs/fern/versions/latest.yml

@@ -361,6 +361,8 @@ navigation:
         contents:
           - page: Agents
             path: ../../studio/agents.mdx
+          - page: Custom Models
+            path: ../../studio/customization.mdx
           - page: Data Designer
             path: ../../studio/data-designer-build.mdx
           - page: Experiments

docs/studio/customization.mdx

@@ -0,0 +1,76 @@
+---
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+title: "NeMo Studio Custom Models"
+description: "Fine-tune models from NeMo Studio, and track customization jobs."
+---
+
+Use **Models > Fine-tune** in the NeMo Studio workspace sidebar to open the Custom Models list, start Customizer jobs, and track their progress for the selected workspace.
+
+<Note>
+
+Custom Models is enabled by default. An administrator can disable it by setting `studio.feature_flags.customizer_enabled: false` in the platform configuration and restarting Studio, which removes the **Custom Models** navigation entry and hides Customizer jobs from the general **Jobs** list.
+
+</Note>
+
+For customization concepts, supported models, and the underlying CLI and API workflow, see [Customization Concepts](/documentation/customizer-reference/customization-concepts).
+
+## Start a Customization Job
+
+Select **Customize a Model** from the Custom Models list, or open a base model's details panel and select **Customize this Model**. Both open **Fine-tune a Model**, a single form with the following sections, shown as they apply to your selections:
+
+| Section | What you configure |
+| ------- | ------------------- |
+| Training Backend | **Automodel** (multi-GPU), **Unsloth** (single-GPU, 4-bit quantized), or **RL**. |
+| Model | Base model, output model name, and description. |
+| Training Method | For Automodel and Unsloth: fine-tuning type (**LoRA**, **LoRA (Merged)**, or **Full Weights**), plus, for Automodel, training type (**SFT** or **Distillation**, with a teacher model field for distillation). For RL: **DPO** or **GRPO**. |
+| Reward Environment | Shown for GRPO jobs. Selects the reward environment used during training. |
+| Dataset | The training fileset, selected from filesets available in the workspace. |
+| Parameters | General hyperparameters, or GRPO-specific parameters when the training method is GRPO. |
+| LoRA Parameters | Shown when the training method uses a LoRA or LoRA-merged finetuning type. |
+| DPO Parameters | Shown for RL jobs using DPO. |
+| Compute Resources | GPU and node allocation for the job. |
+
+Several sections include an advanced JSON field for power-user overrides — for example raw parallelism or backend-specific keyword arguments — alongside the structured form fields. Use these only when a setting isn't exposed elsewhere in the form.
+
+Select **Start Fine-Tuning** to create the job. Studio confirms with a toast and navigates to the new job's detail page.
+
+Model eligibility for fine-tuning depends on the selected base model; **Customize this Model** is disabled for models that have no fileset to fine-tune from.
+
+### Clone an Existing Job
+
+From a job's detail page, open the actions menu (**⋯**) and select **Clone** to open **Fine-tune a Model** pre-filled with that job's settings. Adjust any fields before submitting to start a new job.
+
+## Custom Models List
+
+The Custom Models list shows fine-tuned models created in the current workspace. Select a row to open the model details side panel, which includes **Model Details** and **Chat Playground** tabs, a **Customize this Model** action, an **Evaluate this Model** action, and a **View Intake Traces** link when intake is enabled.
+
+## Customization Job Details
+
+Select a job to open its detail page, which shows the job name, status badge, and metadata such as training type, base model, and creation date.
+
+| Tab | Shows |
+| --- | ----- |
+| Overview | Training progress, run configuration, and the source dataset's fileset details. |
+| Logs | Job execution logs. |
+| Chat with your Model | A chat playground against the resulting model, shown once the job completes. |
+
+The Overview tab's training progress panel differs by training method:
+
+| Training method | Panel | Shows |
+| ---------------- | ----- | ----- |
+| SFT, distillation, DPO | Training Loss | Train and validation loss per step, plus loss and progress stat tiles. |
+| GRPO | Reward | Training and validation reward per step (loss isn't meaningful for GRPO's policy-gradient objective), plus reward and progress stat tiles. |
+| GRPO | Training Health | Collapsed by default. Diagnostic charts from NeMo RL, such as KL divergence and clip fraction, that can flag a run diverging before reward does. |
+
+Below the training panel, **Run Configuration** summarizes the customization ID, output model, base model, and other run metadata, with a **View Job Configuration** action that opens the full job spec.
+
+From the detail page header, select **Evaluate** to start an evaluation against the resulting model once the job is launchable, or open the actions menu (**⋯**) for **Clone** and, while the job is active or pending, **Cancel Job**.
+
+## Related Topics
+
+- [Customization Concepts](/documentation/customizer-reference/customization-concepts)
+- [Create a Customization Job](/documentation/customizer-reference/manage-customization-jobs/create-a-customization-job)
+- [Customization Job Reference](/documentation/customizer-reference/manage-customization-jobs/customization-job-reference)
+- [Experiments](/documentation/evaluate-models/experiments)

docs/studio/index.mdx

@@ -35,7 +35,7 @@ The current sidebar groups related pages under expandable parents. Selecting a p
 | **Observability** | **Insights** | Review recurring agent failure patterns and optimization insights. Shown when optimization is enabled. |
 | **Observability** | **Traces** | Inspect ingested traces, spans, and session details. |
 | **Components** | **Agents** | Opens the agent list. Select an agent to access Deployments, Logs, Chat, Evaluations, and Details. |
-| **Components** | **Models** | Opens the base-model list directly. Expand **Models** for **Fine-tune**, **Model Evaluations**, **Playground**, and **Virtual Models**, when those capabilities are enabled. |
+| **Components** | **Models** | Opens the base-model list directly. Expand **Models** for **Fine-tune**, **Model Evaluations**, **Playground**, and **Virtual Models**, when those capabilities are enabled. **Fine-tune** opens the Custom Models list; see [NeMo Studio Custom Models](/documentation/studio/customization). |
 | **Evaluations** | **Experiments** | Run evaluations against an agent, and review experiments, evaluations, sessions, and published results across the workspace. See [NeMo Studio Experiments](/documentation/studio/experiments). |
 | **Data** | **Datasets > Data Designer** | Build and monitor synthetic-data jobs. Other installed data plugins can appear below **Datasets**. |
 | **Governance** | **Guardrails** | Manage guardrail configurations. Disabled by default; see [Studio Guardrail Configs](/documentation/studio/guardrail-configs). |
@@ -81,6 +81,10 @@ The Base Models page supports searching by model name, filtering by creation or
 
 The model details side panel shows model metadata, parameters, and available actions. From the side panel you can open the chat playground for chat-capable models or navigate to model evaluation results.
 
+### Custom Models
+
+Select **Models > Fine-tune** to open the Custom Models list, start a fine-tuning job against an eligible base model, and track job progress. Enabled by default; see [NeMo Studio Custom Models](/documentation/studio/customization).
+
 ### Workspaces
 
 Keep models and data organized in dedicated workspaces.