Add NIM inference path for the Cosmos 3 Generator#258
Open
ekrivovnv wants to merge 7 commits into
Open
Conversation
nv-asotelo
reviewed
Jul 2, 2026
| | [vLLM](#vllm) | OpenAI-compatible reasoning server (image/video understanding) | Reasoner | | ||
| | [vLLM-Omni](#vllm-omni) | OpenAI-compatible generation server (image/video/audio/action) | Generator (Audiovisual, Action) | | ||
| | [NIM](#nim) | Prebuilt OpenAI-compatible reasoning server (image/video understanding); no venv | Reasoner | | ||
| | [NIM](#nim) | Prebuilt NGC containers; Reasoner serving plus Generator audiovisual T2V/I2V only | Reasoner, Generator (Audiovisual) | |
There was a problem hiding this comment.
Use the same naming conventions as above (image/video/audio). Since this reflects two separate NIMs for now, use (video to text reasoning) for reasoner.
nv-asotelo
reviewed
Jul 2, 2026
| - [`uv`](https://docs.astral.sh/uv/getting-started/installation/), `git`, and `git-lfs` installed. | ||
| - Hugging Face access to the gated Cosmos3 model repos. Generator also requires | ||
| access to the gated | ||
| - Hugging Face access to the gated Cosmos3 model repos for Hugging Face based |
There was a problem hiding this comment.
be more explicit, or state all supported backends except NIM.
Make sure to remember SGLang and TRT-LLM if making access claims.
nv-asotelo
reviewed
Jul 2, 2026
|
|
||
| A prebuilt container that serves **Cosmos3-Generator Text2Video and Image2Video | ||
| only** through `POST /v1/infer`. The NIM infers mode from the request fields: | ||
| non-empty `prompt` with no `image` means T2V; `image` provided means I2V. The |
There was a problem hiding this comment.
Explicit - Text-to-video, image-to-video
nv-asotelo
reviewed
Jul 2, 2026
| - **World generation:** Produce images, videos, synchronized sound, and action-conditioned rollouts from text, image, video, or action inputs. | ||
| - **Action modeling:** Predict policy actions, inverse dynamics, and forward dynamics for robotics, camera motion, egocentric motion, and autonomous-driving settings. | ||
| - **Research and production paths:** Use Diffusers and Transformers for Python-first development, then vLLM-Omni and vLLM for OpenAI-compatible serving. | ||
| - **Research and production paths:** Use Diffusers and Transformers for Python-first development, vLLM-Omni and vLLM for OpenAI-compatible serving, and NIM containers for turnkey Reasoner serving plus Generator T2V/I2V deployment. |
There was a problem hiding this comment.
explicitly, "text to video" and "image to video". This is a different readme file and the first time the acronym is presented.
|
@ekrivovnv I added some comments, just minor nits. Please address and LGTM |
Author
|
@nv-asotelo I think all your comments were addressed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added tutorial on using NIM for the Generator path.
Also, added benchmarks for the Generator, overall they show perfomance improvement over vllm-omni.