Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Triton 24.04 #1738

Open
wants to merge 5 commits into
base: branch-24.06
Choose a base branch
from

Conversation

mdemoret-nv
Copy link
Contributor

Description

  • Upgrade Triton from 22.10 or 23.06 in places to 24.04 across the board
  • Updates the utilities scripts to use the correct version and updates the submodule
  • Updates all references to launching triton in the docs

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

@mdemoret-nv mdemoret-nv added non-breaking Non-breaking change improvement Improvement to existing functionality labels Jun 14, 2024
@mdemoret-nv mdemoret-nv requested review from a team as code owners June 14, 2024 17:24
```

##### Start Triton Inference Server Container
From the Morpheus repo root directory, run the following to launch Triton and load the `log-parsing-onnx` model:

```bash
docker run --rm -ti --gpus=all -p8000:8000 -p8001:8001 -p8002:8002 -v $PWD/models:/models nvcr.io/nvidia/tritonserver:23.06-py3 tritonserver --model-repository=/models/triton-model-repo --exit-on-error=false --model-control-mode=explicit --load-model log-parsing-onnx
docker run --rm -ti --gpus=all -p8000:8000 -p8001:8001 -p8002:8002 -v $PWD/models:/models nvcr.io/nvidia/tritonserver:24.04-py3 tritonserver --model-repository=/models/triton-model-repo --exit-on-error=false --model-control-mode=explicit --load-model log-parsing-onnx
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating Triton version here breaks log parsing as described in #1727. The output config would have to be updated as follows:

output [
    {
        name: "output"
        data_type: TYPE_FP32
        dims: [ 256, 23 ]
    }
]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement to existing functionality non-breaking Non-breaking change
Projects
Status: Review - Ready for Review
Development

Successfully merging this pull request may close these issues.

2 participants