File tree Expand file tree Collapse file tree 1 file changed +19
-8
lines changed Expand file tree Collapse file tree 1 file changed +19
-8
lines changed Original file line number Diff line number Diff line change @@ -103,23 +103,23 @@ make install
103103
104104### PyTorch 2.0 Models
105105
106+ PyTorch 2.0 features are available.
107+ However, Triton's PyTorch backend requires a serialized representation of the model in the form a ` model.pt ` file.
108+ The serialized representation of the model can be generated using PyTorch's
109+ [ ` torch.save() ` ] ( https://docs.pytorch.org/tutorials/beginner/saving_loading_models.html#id1 )
110+ function to generate the ` model.pt ` file.
111+
106112The model repository should look like:
107113
108114``` bash
109115model_repository/
110116` -- model_directory
111117 | -- 1
112- | | -- model.py
113- | ` -- [model.pt]
118+ | ` -- model.pt
114119 ` -- config.pbtxt
115120` ` `
116121
117- The ` model.py` contains the class definition of the PyTorch model.
118- The class should extend the
119- [` torch.nn.Module` ](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module).
120- The ` model.pt` may be optionally provided which contains the saved
121- [` state_dict` ](https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-model-for-inference)
122- of the model.
122+ Where ` model.pt` is the serialized representation of the model.
123123
124124# ## TorchScript Models
125125
@@ -139,6 +139,17 @@ The `model.pt` is the TorchScript model file.
139139
140140Triton exposes some flags to control the execution mode of the TorchScript models through the ` Parameters ` section of the model's ` config.pbtxt ` file.
141141
142+ ### Configuration Options
143+
144+ * ` default_model_name ` :
145+ Instructs the Triton PyTorch backend to load the model from a file of the given name.
146+
147+ The model config specifying the option would look like:
148+
149+ ``` proto
150+ default_model_name: "another_file_name.pt"
151+ ```
152+
142153### Parameters
143154
144155* ` DISABLE_OPTIMIZED_EXECUTION ` :
You can’t perform that action at this time.
0 commit comments