Skip to content

Commit

Permalink
Add a base README for list_plugins tool
Browse files Browse the repository at this point in the history
  • Loading branch information
rmccorm4 committed Mar 13, 2020
1 parent c2cbbd7 commit b65c5de
Show file tree
Hide file tree
Showing 3 changed files with 96 additions and 1 deletion.
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,8 @@ resnet50/
# TensorRT Engines
*.plan
*.engine

# Plugins
*.d
*.o
*.so
91 changes: 91 additions & 0 deletions plugins/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Plugins

This directory contains some tools and samples related to
TensorRT Plugins.

## Listing Registered Plugins

```
$ python list_plugins.py -h
usage: list_plugins.py [-h] [-p [PLUGINS [PLUGINS ...]]]
Script to list registered TensorRT plugins. Can optionally load custom plugin
libraries.
optional arguments:
-h, --help show this help message and exit
-p [PLUGINS [PLUGINS ...]], --plugins [PLUGINS [PLUGINS ...]]
Path to a plugin (.so) library file. Accepts multiple
arguments.
```

Default `TRT_RELEASE` plugins:
```
$ python list_plugins.py
2020-03-13 22:12:35 - __main__ - INFO - Registering plugins...
2020-03-13 22:12:35 - __main__ - INFO - Registered Plugin Names:
['RnRes2Br2bBr2c_TRT',
'RnRes2Br1Br2c_TRT',
'CgPersistentLSTMPlugin_TRT',
'SingleStepLSTMPlugin',
...
'SpecialSlice_TRT',
'InstanceNormalization_TRT']
```

## Registering OSS Plugins

When building the OSS components, the default plugin library may be overwritten
or given precendence to the OSS plugin library. As long as you have both .so
library files, you can load both:

Build OSS Components:
```
$ wget https://raw.githubusercontent.com/rmccorm4/tensorrt-utils/master/OSS/build_OSS.sh
$ source build_OSS.sh
```

Load and list multiple plugin libraries:
```
$ python list_plugins.py --plugins TensorRT/build/out/libnvinfer_plugin.so
2020-03-13 22:18:51 - __main__ - INFO - Loading plugin library: /mnt/TensorRT/build/out/libnvinfer_plugin.so
2020-03-13 22:18:51 - __main__ - INFO - Registering plugins...
2020-03-13 22:18:51 - __main__ - INFO - Registered Plugin Names:
['RnRes2Br2bBr2c_TRT',
'RnRes2Br1Br2c_TRT',
'CgPersistentLSTMPlugin_TRT',
'SingleStepLSTMPlugin',
'CustomEmbLayerNormPluginDynamic', <------ OSS Plugins added
'CustomFCPluginDynamic', <------ OSS Plugins added
'CustomGeluPluginDynamic', <------ OSS Plugins added
'CustomQKVToContextPluginDynamic', <------ OSS Plugins added
'CustomSkipLayerNormPluginDynamic', <------ OSS Plugins added
...
'SpecialSlice_TRT',
'InstanceNormalization_TRT']
```

## Registering Custom Plugins

Same concept as above for OSS plugins, just giving an example to be extra clear:
```
$ pushd CustomIPluginV2/
$ make
g++ -g -std=c++11 -DNDEBUG -fPIC -MD -MP -I. -I/usr/local/cuda/include -I/usr/src/tensorrt/include -o CustomPlugin.o -c CustomPlugin.cpp
g++ -g -std=c++11 -DNDEBUG -shared -o CustomPlugin.so CustomPlugin.o -L/usr/local/cuda/lib64 -L/usr/src/tensorrt/lib -lnvinfer -lcudart
$ popd
$ python list_plugins.py --plugins CustomIPluginV2/CustomPlugin.so
2020-03-13 22:16:06 - __main__ - INFO - Loading plugin library: CustomIPluginV2/CustomPlugin.so
Plugin attribute number: 5
2020-03-13 22:16:06 - __main__ - INFO - Registering plugins...
2020-03-13 22:16:06 - __main__ - INFO - Registered Plugin Names:
['RnRes2Br2bBr2c_TRT',
'RnRes2Br1Br2c_TRT',
'CgPersistentLSTMPlugin_TRT',
'SingleStepLSTMPlugin',
'CustomPlugin', <------------- Custom plugin registered
...
'SpecialSlice_TRT',
'InstanceNormalization_TRT']
```
1 change: 0 additions & 1 deletion plugins/list_plugins.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,6 @@ def get_all_plugin_names(plugin_registry):
parser.add_argument("-p", "--plugins", nargs="*", default=[], help="Path to a plugin (.so) library file. Accepts multiple arguments.")
args = parser.parse_args()

print(args.plugins)
for plugin_library in args.plugins:
# Example default plugin library: "/usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so"
logger.info("Loading plugin library: {}".format(plugin_library))
Expand Down

0 comments on commit b65c5de

Please sign in to comment.