Skip to content

Commit 92692f8

Browse files
authored
docs: Correct README Instructions (#164)
This change corrects the instruction for how to use PyTorch 2 with the backend.
1 parent 6d02d35 commit 92692f8

File tree

1 file changed

+19
-8
lines changed

1 file changed

+19
-8
lines changed

README.md

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -103,23 +103,23 @@ make install
103103

104104
### PyTorch 2.0 Models
105105

106+
PyTorch 2.0 features are available.
107+
However, Triton's PyTorch backend requires a serialized representation of the model in the form a `model.pt` file.
108+
The serialized representation of the model can be generated using PyTorch's
109+
[`torch.save()`](https://docs.pytorch.org/tutorials/beginner/saving_loading_models.html#id1)
110+
function to generate the `model.pt` file.
111+
106112
The model repository should look like:
107113

108114
```bash
109115
model_repository/
110116
`-- model_directory
111117
|-- 1
112-
| |-- model.py
113-
| `-- [model.pt]
118+
| `-- model.pt
114119
`-- config.pbtxt
115120
```
116121

117-
The `model.py` contains the class definition of the PyTorch model.
118-
The class should extend the
119-
[`torch.nn.Module`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module).
120-
The `model.pt` may be optionally provided which contains the saved
121-
[`state_dict`](https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-model-for-inference)
122-
of the model.
122+
Where `model.pt` is the serialized representation of the model.
123123

124124
### TorchScript Models
125125

@@ -139,6 +139,17 @@ The `model.pt` is the TorchScript model file.
139139

140140
Triton exposes some flags to control the execution mode of the TorchScript models through the `Parameters` section of the model's `config.pbtxt` file.
141141

142+
### Configuration Options
143+
144+
* `default_model_name`:
145+
Instructs the Triton PyTorch backend to load the model from a file of the given name.
146+
147+
The model config specifying the option would look like:
148+
149+
```proto
150+
default_model_name: "another_file_name.pt"
151+
```
152+
142153
### Parameters
143154

144155
* `DISABLE_OPTIMIZED_EXECUTION`:

0 commit comments

Comments
 (0)