Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/agents/deploy-agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,7 @@ models:
```

To make an external model available to the agent, register a provider first —
see [Deploy Models](/documentation/models-and-inference/tutorials/deploy-models#add-external-providers)
see [Deploy Models](/documentation/models-and-inference/deploy-models#add-external-providers)
for NVIDIA Build, OpenAI, and Anthropic examples.

<Note>
Expand Down
2 changes: 1 addition & 1 deletion docs/customizer/grpo-training.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@ GRPO fails at submit, before any GPU is claimed, unless the platform operator ha

If a submit fails on either of the first two, that is a platform configuration gap, not a problem with your package. Neither is settable per job.

Installing OpenSandbox is a separate operator task: [OpenSandbox](/documentation/kubernetes-deployment/setup/helm/opensandbox), or [OpenSandbox with Kata](/documentation/kubernetes-deployment/setup/helm/opensandbox-kata) for the Kata runtime.
Installing OpenSandbox is a separate operator task: [OpenSandbox](/documentation/kubernetes-deployment/setup/helm/open-sandbox), or [OpenSandbox with Kata](/documentation/kubernetes-deployment/setup/helm/opensandbox-kata) for the Kata runtime.

## Next Steps

Expand Down
4 changes: 2 additions & 2 deletions docs/customizer/tutorials/grpo-environment-packages.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Do not put `.jsonl` in the environment package. If a Gym config lists `datasets[
Before packaging an environment, ensure you have:

1. **A NeMo Platform configured with `platform.runtime: kubernetes`** — GRPO provisions a Ray cluster and has no local Docker fallback
2. **Sandboxed Gym enabled on the cluster** by your platform operator (`sandbox_cluster_capable` and a job-storage PVC claim). Refer to [Cluster prerequisites](/documentation/customizer-reference/grpo-and-reward-environments#cluster-prerequisites); installing OpenSandbox is covered in [OpenSandbox](/documentation/kubernetes-deployment/setup/helm/opensandbox)
2. **Sandboxed Gym enabled on the cluster** by your platform operator (`sandbox_cluster_capable` and a job-storage PVC claim). Refer to [Cluster prerequisites](/documentation/customizer-reference/grpo-and-reward-environments#cluster-prerequisites); installing OpenSandbox is covered in [OpenSandbox](/documentation/kubernetes-deployment/setup/helm/open-sandbox)
3. **The `nemo` CLI on your `PATH`**, or a source checkout where `uv run --package nmp-rl` resolves
4. **The training image tag** your cluster runs — vendoring a wheel closure requires the `nemo-gym`, `ray`, and `openai` versions it reports
5. **A machine with internet access** for the packaging step. Training clusters consume uploaded FileSets only. For details on which environment type require internet access too, see details below.
Expand Down Expand Up @@ -430,7 +430,7 @@ nemo files upload ./weather-env/ weather-env --workspace default
nemo files list weather-env --workspace default
```

Job JSON: [GRPO and Reward Environments](/documentation/customizer-reference/grpo-and-reward-environments#submit-the-job). Sandboxed Gym is configured on the platform (`NMP_RL_SANDBOXED_GYM_DEFAULT`), not in the job payload. Cluster setup: [OpenSandbox](/documentation/kubernetes-deployment/setup/helm/opensandbox).
Job JSON: [GRPO and Reward Environments](/documentation/customizer-reference/grpo-and-reward-environments#submit-the-job). Sandboxed Gym is configured on the platform (`NMP_RL_SANDBOXED_GYM_DEFAULT`), not in the job payload. Cluster setup: [OpenSandbox](/documentation/kubernetes-deployment/setup/helm/open-sandbox).

## Next Steps

Expand Down
23 changes: 23 additions & 0 deletions docs/fern/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,15 @@ redirects:
# Sphinx page URLs (foo.html); must follow all :path*/index.html rules above
- source: "/:path*.html"
destination: "/:path*"
# NV 6722631: OpenSandbox how-to live slug is open-sandbox.
- source: "/documentation/self-managed-deployment/setup/helm/opensandbox"
destination: "/documentation/kubernetes-deployment/setup/helm/open-sandbox"
- source: "/latest/documentation/self-managed-deployment/setup/helm/opensandbox"
destination: "/latest/documentation/kubernetes-deployment/setup/helm/open-sandbox"
- source: "/documentation/kubernetes-deployment/setup/helm/opensandbox"
destination: "/documentation/kubernetes-deployment/setup/helm/open-sandbox"
- source: "/latest/documentation/kubernetes-deployment/setup/helm/opensandbox"
destination: "/latest/documentation/kubernetes-deployment/setup/helm/open-sandbox"
# NV 6722623: published hrefs still use the old self-managed-deployment nav slug.
- source: "/documentation/self-managed-deployment/helm"
destination: "/documentation/kubernetes-deployment/setup/helm"
Expand All @@ -45,3 +54,17 @@ redirects:
destination: "/documentation/kubernetes-deployment/:path*"
- source: "/latest/documentation/self-managed-deployment/:path*"
destination: "/latest/documentation/kubernetes-deployment/:path*"
# NV 6722632: Models and Inference pages are no longer under /tutorials/.
- source: "/documentation/models-and-inference/tutorials/run-inference"
destination: "/documentation/models-and-inference/run-inference"
- source: "/latest/documentation/models-and-inference/tutorials/run-inference"
destination: "/latest/documentation/models-and-inference/run-inference"
- source: "/documentation/models-and-inference/tutorials/deploy-models"
destination: "/documentation/models-and-inference/deploy-models"
- source: "/latest/documentation/models-and-inference/tutorials/deploy-models"
destination: "/latest/documentation/models-and-inference/deploy-models"
# NV 6722634: Plugin UIs title otherwise kebab-cases to plugin-u-is.
- source: "/documentation/studio/plugin-u-is"
destination: "/documentation/studio/plugins"
- source: "/latest/documentation/studio/plugin-u-is"
destination: "/latest/documentation/studio/plugins"
28 changes: 25 additions & 3 deletions docs/fern/scripts/sync-helm-docs.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,29 @@ const DROP_LINE =
/^Documentation can be found at: https:\/\/docs\.nvidia\.com\/nemo-platform\.\s*$/;

const DEPLOYMENT_LINK =
/For deployment instructions, see https:\/\/docs\.nvidia\.com\/nemo-platform\/documentation\/self-managed-deployment\/setup\./;
/For deployment instructions, see https:\/\/docs\.nvidia\.com\/nemo-platform\/(?:latest\/)?documentation\/(?:self-managed-deployment|kubernetes-deployment)\/setup\.?/;

const DEPLOYMENT_LINK_REPLACEMENT =
"For deployment guide, see " +
"[Self-Managed Deployment](/documentation/self-managed-deployment/setup) " +
"[Kubernetes Deployment](/documentation/kubernetes-deployment/setup) " +
"in the NeMo Platform documentation.";

const EXAMPLES_OVERLAY_LINK = /\[examples\/opensandbox\]\(examples\/opensandbox\)/;
const EXAMPLES_OVERLAY_REPLACEMENT =
"[k8s/helm/examples/opensandbox](https://github.com/NVIDIA-NeMo/nemo-platform/tree/main/k8s/helm/examples/opensandbox)";

function rewritePublishedDocsUrls(text) {
return text
.replace(
/\/documentation\/self-managed-deployment\//g,
"/documentation/kubernetes-deployment/",
)
.replace(
/(\/documentation\/kubernetes-deployment\/setup\/helm\/)opensandbox(?!-kata)/g,
"$1open-sandbox",
);
}

function stripLeadingSpdxComments(markdown) {
const lines = markdown.split("\n");

Expand Down Expand Up @@ -68,7 +84,13 @@ function extractIntro(readme) {

if (DROP_LINE.test(line)) continue;

out.push(line.replace(DEPLOYMENT_LINK, DEPLOYMENT_LINK_REPLACEMENT));
out.push(
rewritePublishedDocsUrls(
line
.replace(DEPLOYMENT_LINK, DEPLOYMENT_LINK_REPLACEMENT)
.replace(EXAMPLES_OVERLAY_LINK, EXAMPLES_OVERLAY_REPLACEMENT),
),
);
}

while (out.length && !out[out.length - 1].trim()) out.pop();
Expand Down
2 changes: 2 additions & 0 deletions docs/fern/versions/latest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,7 @@ navigation:
- page: Monitor
path: ../../studio/monitor.mdx
- page: Plugin UIs
slug: plugins
path: ../../studio/plugins.mdx
- page: Guardrail Configs
path: ../../studio/guardrails.mdx
Expand Down Expand Up @@ -390,6 +391,7 @@ navigation:
- page: Volcano
path: ../../set-up/helm/volcano.mdx
- page: OpenSandbox
slug: open-sandbox
path: ../../set-up/helm/opensandbox.mdx
- page: OpenSandbox with Kata
slug: opensandbox-kata
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,4 +137,4 @@ client.inference.virtual_models.delete(name="guarded-with-defaults")
- [Architecture](/documentation/guardrail-models/core-concepts/architecture) — Understand the middleware pipeline and VirtualModel wiring
- [Configuration Structure](/documentation/guardrail-models/core-concepts/configurations/configuration-structure) — Configuration schema reference
- [Manage Configurations](/documentation/guardrail-models/core-concepts/configurations/manage-configurations) — Create and manage your own configurations
- [Run Inference](/documentation/models-and-inference/tutorials/run-inference) — Run inference with a guardrail configuration
- [Run Inference](/documentation/models-and-inference/run-inference) — Run inference with a guardrail configuration
1 change: 1 addition & 0 deletions docs/run-inference/tutorials/deploy-models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Resource names for deployments, deployment configs, and providers must contain o
<Markdown src="/snippets/_snippets/tutorials/cli-sdk-setup.mdx" />
---

<a id="add-external-providers"></a>
## Add External Providers

Register external inference APIs like NVIDIA Build or OpenAI.
Expand Down
2 changes: 1 addition & 1 deletion docs/run-inference/tutorials/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ Learn how to run inference through the NeMo Platform.

## Guides

- [Run Inference](/documentation/models-and-inference/tutorials/run-inference) — Route requests via model entity, provider, or OpenAI routing
- [Run Inference](/documentation/models-and-inference/run-inference) — Route requests via model entity, provider, or OpenAI routing
2 changes: 1 addition & 1 deletion docs/set-up/helm/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ Install the Volcano scheduler for multi-node jobs. The chart does not install Vo
<small><span class="md-tag">cluster-admin</span></small>

</Card>
<Card title="OpenSandbox" href="/documentation/kubernetes-deployment/setup/helm/opensandbox">
<Card title="OpenSandbox" href="/documentation/kubernetes-deployment/setup/helm/open-sandbox">

Install OpenSandbox with the shared-kernel / default runtime path and point jobs at the in-cluster server.

Expand Down
2 changes: 1 addition & 1 deletion docs/set-up/helm/install.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ The chart does not install these. Install them before the workloads that need th

- [Volcano](/documentation/kubernetes-deployment/setup/helm/volcano), required for multi-node scheduling
- [Kyverno](/documentation/kubernetes-deployment/setup/helm/multinode-networking), required for multi-node NCCL device injection
- [OpenSandbox](/documentation/kubernetes-deployment/setup/helm/opensandbox) required for sandboxed GRPO; [Kata](/documentation/kubernetes-deployment/setup/helm/opensandbox-kata) for VM-isolated sandboxes
- [OpenSandbox](/documentation/kubernetes-deployment/setup/helm/open-sandbox) required for sandboxed GRPO; [Kata](/documentation/kubernetes-deployment/setup/helm/opensandbox-kata) for VM-isolated sandboxes

## Install the chart

Expand Down
2 changes: 1 addition & 1 deletion docs/set-up/helm/opensandbox-kata.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: "Install Kata Containers and an OpenSandbox server that pins sandbo
---
<a id="install-opensandbox-kata"></a>

Use this path when **sandboxed GRPO / NeMo Gym** sandbox pods must be isolated from the host kernel. We document **Kata QEMU** (`RuntimeClass/kata-qemu`) because it runs each sandbox in a QEMU VM with its own guest kernel. Other kernel-isolated runtimes (for example Kata Firecracker, or a different hardware-isolated RuntimeClass) may work the same way, but they have **not been tested** with NeMo Platform. **Kata is optional**; most installs only need the [shared-kernel OpenSandbox](/documentation/kubernetes-deployment/setup/helm/opensandbox) path. This does **not** sandbox the rest of the platform (API, DPO, SFT, inference).
Use this path when **sandboxed GRPO / NeMo Gym** sandbox pods must be isolated from the host kernel. We document **Kata QEMU** (`RuntimeClass/kata-qemu`) because it runs each sandbox in a QEMU VM with its own guest kernel. Other kernel-isolated runtimes (for example Kata Firecracker, or a different hardware-isolated RuntimeClass) may work the same way, but they have **not been tested** with NeMo Platform. **Kata is optional**; most installs only need the [shared-kernel OpenSandbox](/documentation/kubernetes-deployment/setup/helm/open-sandbox) path. This does **not** sandbox the rest of the platform (API, DPO, SFT, inference).

Complete the OpenSandbox namespace, Secret, and platform-values steps first; this page only adds virtualization prerequisites and the Kata server overlay.

Expand Down
14 changes: 7 additions & 7 deletions k8s/helm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square)

Documentation can be found at: https://docs.nvidia.com/nemo-platform.
For deployment instructions, see https://docs.nvidia.com/nemo-platform/documentation/self-managed-deployment/setup.
For deployment instructions, see https://docs.nvidia.com/nemo-platform/documentation/kubernetes-deployment/setup.

## Platform Secrets Encryption Key

Expand Down Expand Up @@ -124,15 +124,15 @@ and
The chart does not install Kyverno. Multi-node NCCL device injection renders
ClusterPolicies that Kyverno must apply. Enable exactly one cloud provider
under `multinodeNetworking`.
How-to: https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/multinode-networking
How-to: https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/multinode-networking

## Volcano

The chart does not install Volcano. Multi-node `volcano_job` workloads need it.
`rbac.volcanoEnabled` defaults to true so the core controller can manage Volcano
CRs. Skip Volcano and set `rbac.volcanoEnabled: false` if you are not running
those jobs.
How-to: https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/volcano
How-to: https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/volcano

## OpenSandbox

Expand All @@ -144,9 +144,9 @@ already installed server as an HTTP client (`OPEN_SANDBOX_DOMAIN`,
namespace**. Control plane may stay in `opensandbox-system`. Copy the API-key
Secret into the job namespace.

Example overlays: [examples/opensandbox](examples/opensandbox).
Shared-kernel (cluster default OCI runtime): https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/opensandbox
Kata QEMU: https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/opensandbox-kata
Example overlays: [k8s/helm/examples/opensandbox](https://github.com/NVIDIA-NeMo/nemo-platform/tree/main/k8s/helm/examples/opensandbox).
Shared-kernel (cluster default OCI runtime): https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/open-sandbox
Kata QEMU: https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/opensandbox-kata

## NetworkPolicies

Expand Down Expand Up @@ -437,7 +437,7 @@ For the complete default values, see [values.yaml](values.yaml).
| ncclTest.iterations | int | `3` | How many times to run the full multinode NCCL test (orchestrator loop; env NCCL_TEST_ITERATIONS). Increase the test timeout on helm test if increasing this variable |
| ncclTest.validation.minBandwidthMBpsAt1024MB | int | `8000` | Minimum allreduce bandwidth (MB/s) at 1024MB message size; 0 disables the floor check in nccl_test.py. |
| ncclTest.waitTimeoutSeconds | int | `900` | Max seconds to wait for each worker pod to complete. |
| networkPolicies | object | [See values.yaml](values.yaml#L143) | NetworkPolicy configuration. Enable the top-level switch to render all default policies, then disable individual policies only for cluster-specific exceptions. For a Calico-backed smoke test, see https://docs.nvidia.com/nemo-platform/documentation/self-managed-deployment/setup/helm/network-policy-smoke-test. |
| networkPolicies | object | [See values.yaml](values.yaml#L143) | NetworkPolicy configuration. Enable the top-level switch to render all default policies, then disable individual policies only for cluster-specific exceptions. For a Calico-backed smoke test, see https://docs.nvidia.com/nemo-platform/documentation/kubernetes-deployment/setup/helm/network-policy-smoke-test. |
| networkPolicies.api | object | [See values.yaml](values.yaml#L148) | NetworkPolicy configuration for the Platform API pods. |
| networkPolicies.api.enabled | bool | `true` | Create NetworkPolicy resources that isolate Platform API pod ingress. |
| networkPolicies.api.extraIngress | list | `[]` | Extra NetworkPolicy ingress rules appended to the API policy, for cluster-specific ingress controllers, gateways, monitoring, or debugging pods. |
Expand Down
4 changes: 2 additions & 2 deletions k8s/helm/examples/opensandbox/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ sandbox pods must be isolated from the host kernel. The documented example is
Kata QEMU because it runs each sandbox in a VM with its own guest kernel; other
isolated runtimes may work but have not been tested.

Full procedure: [OpenSandbox](https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/opensandbox)
and [OpenSandbox with Kata](https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/opensandbox-kata)
Full procedure: [OpenSandbox](https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/open-sandbox)
and [OpenSandbox with Kata](https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/opensandbox-kata)
in the NeMo Platform documentation. `helm show readme` of this chart points at
those pages.

Expand Down
12 changes: 6 additions & 6 deletions k8s/helm/helm-docs-template/nemo-helm-readme.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
{{ template "chart.typeBadge" . }}

Documentation can be found at: https://docs.nvidia.com/nemo-platform.
For deployment instructions, see https://docs.nvidia.com/nemo-platform/documentation/self-managed-deployment/setup.
For deployment instructions, see https://docs.nvidia.com/nemo-platform/documentation/kubernetes-deployment/setup.

## Platform Secrets Encryption Key

Expand Down Expand Up @@ -124,15 +124,15 @@ and
The chart does not install Kyverno. Multi-node NCCL device injection renders
ClusterPolicies that Kyverno must apply. Enable exactly one cloud provider
under `multinodeNetworking`.
How-to: https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/multinode-networking
How-to: https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/multinode-networking

## Volcano

The chart does not install Volcano. Multi-node `volcano_job` workloads need it.
`rbac.volcanoEnabled` defaults to true so the core controller can manage Volcano
CRs. Skip Volcano and set `rbac.volcanoEnabled: false` if you are not running
those jobs.
How-to: https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/volcano
How-to: https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/volcano

## OpenSandbox

Expand All @@ -144,9 +144,9 @@ already installed server as an HTTP client (`OPEN_SANDBOX_DOMAIN`,
namespace**. Control plane may stay in `opensandbox-system`. Copy the API-key
Secret into the job namespace.

Example overlays: [examples/opensandbox](examples/opensandbox).
Shared-kernel (cluster default OCI runtime): https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/opensandbox
Kata QEMU: https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/opensandbox-kata
Example overlays: [k8s/helm/examples/opensandbox](https://github.com/NVIDIA-NeMo/nemo-platform/tree/main/k8s/helm/examples/opensandbox).
Shared-kernel (cluster default OCI runtime): https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/open-sandbox
Kata QEMU: https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/opensandbox-kata

## NetworkPolicies

Expand Down
2 changes: 1 addition & 1 deletion k8s/helm/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,6 @@ Successfully installed {{ .Chart.Name }}-{{ .Chart.Version }}, named {{ .Release
OPEN_SANDBOX_DOMAIN={{ .Values.opensandbox.domain }} over {{ .Values.opensandbox.protocol }}.
Copy Secret {{ .Values.opensandbox.apiKeySecret }} into this release namespace.
The chart does not install OpenSandbox. See helm show readme and
https://docs.nvidia.com/nemo-platform/latest/documentation/self-managed-deployment/setup/helm/opensandbox
https://docs.nvidia.com/nemo-platform/latest/documentation/kubernetes-deployment/setup/helm/open-sandbox

{{- end }}
Loading
Loading