deploy the yolov3 model to TensorRT #1422

emiya777 · 2022-11-23T09:14:26Z

emiya777
Nov 23, 2022

Hi,all.
i tried to deploy the yolov3 model to TensorRT,and i build the mmdeploy env with docker in win10.
i got onnx file, but but it doesn't work.Environment and error log below

Environment

2022-11-23 08:33:37,029 - mmdeploy - INFO -

2022-11-23 08:33:37,029 - mmdeploy - INFO - **********Environmental information**********
2022-11-23 08:33:37,242 - mmdeploy - INFO - sys.platform: linux
2022-11-23 08:33:37,242 - mmdeploy - INFO - Python: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0]
2022-11-23 08:33:37,242 - mmdeploy - INFO - CUDA available: True
2022-11-23 08:33:37,242 - mmdeploy - INFO - GPU 0: NVIDIA GeForce GTX 1660
2022-11-23 08:33:37,242 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda
2022-11-23 08:33:37,242 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 11.6, V11.6.124
2022-11-23 08:33:37,242 - mmdeploy - INFO - GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
2022-11-23 08:33:37,242 - mmdeploy - INFO - PyTorch: 1.10.0
2022-11-23 08:33:37,242 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.2
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

2022-11-23 08:33:37,242 - mmdeploy - INFO - TorchVision: 0.11.0
2022-11-23 08:33:37,242 - mmdeploy - INFO - OpenCV: 4.6.0
2022-11-23 08:33:37,243 - mmdeploy - INFO - MMCV: 1.5.3
2022-11-23 08:33:37,243 - mmdeploy - INFO - MMCV Compiler: GCC 7.3
2022-11-23 08:33:37,243 - mmdeploy - INFO - MMCV CUDA Compiler: 11.3
2022-11-23 08:33:37,243 - mmdeploy - INFO - MMDeploy: 0.10.0+522fcc0
2022-11-23 08:33:37,243 - mmdeploy - INFO -

2022-11-23 08:33:37,243 - mmdeploy - INFO - **********Backend information**********
2022-11-23 08:33:38,040 - mmdeploy - INFO - onnxruntime: 1.8.1  ops_is_avaliable : True
2022-11-23 08:33:38,092 - mmdeploy - INFO - tensorrt: 8.2.4.2   ops_is_avaliable : True
2022-11-23 08:33:38,120 - mmdeploy - INFO - ncnn: None  ops_is_avaliable : False
2022-11-23 08:33:38,121 - mmdeploy - INFO - pplnn_is_avaliable: False
2022-11-23 08:33:38,121 - mmdeploy - INFO - openvino_is_avaliable: False
2022-11-23 08:33:38,148 - mmdeploy - INFO - snpe_is_available: False
2022-11-23 08:33:38,149 - mmdeploy - INFO - ascend_is_available: False
2022-11-23 08:33:38,150 - mmdeploy - INFO - coreml_is_available: False
2022-11-23 08:33:38,150 - mmdeploy - INFO -

2022-11-23 08:33:38,150 - mmdeploy - INFO - **********Codebase information**********
2022-11-23 08:33:38,152 - mmdeploy - INFO - mmdet:      2.25.3
2022-11-23 08:33:38,152 - mmdeploy - INFO - mmseg:      None
2022-11-23 08:33:38,152 - mmdeploy - INFO - mmcls:      None
2022-11-23 08:33:38,152 - mmdeploy - INFO - mmocr:      None
2022-11-23 08:33:38,152 - mmdeploy - INFO - mmedit:     None
2022-11-23 08:33:38,152 - mmdeploy - INFO - mmdet3d:    None
2022-11-23 08:33:38,152 - mmdeploy - INFO - mmpose:     None
2022-11-23 08:33:38,153 - mmdeploy - INFO - mmrotate:   None
2022-11-23 08:33:38,153 - mmdeploy - INFO - mmaction:   None

then i execute

python ./tools/deploy.py configs/mmdet/detection/detection_tensorrt-fp16_dynamic-64x64-608x608.py  \ 
    ../v_data/yolov3_dep/go_yolov3_1121.py \
    ../v_data/yolov3_dep/epoch_273.pth \ 
    ../v_data/yolov3_dep/image2_01150.png \
    --work-dir work_dirs/yolov3_to_tensorrt --device cuda:0 --dump-info

i got onnx and engine file successfully and some output

/opt/conda/lib/python3.8/site-packages/mmdet/datasets/utils.py:66: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  warnings.warn(
2022-11-23 08:10:51,441 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
load checkpoint from local path: ../v_data/yolov3_dep/epoch_273.pth
/opt/conda/lib/python3.8/site-packages/mmdet/datasets/utils.py:66: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  warnings.warn(
2022-11-23 08:11:33,834 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
2022-11-23 08:11:33,835 - mmdeploy - INFO - Export PyTorch model to ONNX: work_dirs/yolov3_to_tensorrt/end2end.onnx.
2022-11-23 08:11:33,887 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_deduplicate_initializers, function rewrite will not be applied
/root/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:3631: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/mmdet/models/dense_heads/yolo_head.py:127: UserWarning: DeprecationWarning: `anchor_generator` is deprecated, please use "prior_generator" instead
  warnings.warn('DeprecationWarning: `anchor_generator` is deprecated, '
/opt/conda/lib/python3.8/site-packages/mmdet/core/anchor/anchor_generator.py:333: UserWarning: ``grid_anchors`` would be deprecated soon. Please use ``grid_priors``
  warnings.warn('``grid_anchors`` would be deprecated soon. '
/opt/conda/lib/python3.8/site-packages/mmdet/core/anchor/anchor_generator.py:369: UserWarning: ``single_level_grid_anchors`` would be deprecated soon. Please use ``single_level_grid_priors``
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/mmdet/core/bbox/coder/yolo_bbox_coder.py:73: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(-1) == bboxes.size(-1) == 4
/root/workspace/mmdeploy/mmdeploy/pytorch/functions/topk.py:57: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if k > size:
/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:261: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/root/workspace/mmdeploy/mmdeploy/mmcv/ops/nms.py:178: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  out_boxes = min(num_boxes, after_topk)
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
2022-11-23 08:11:46,969 - mmdeploy - INFO - Execute onnx optimize passes.
2022-11-23 08:11:49,338 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
2022-11-23 08:11:52,440 - mmdeploy - INFO - Start pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt in subprocess
2022-11-23 08:11:52,705 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
[11/23/2022-08:11:53] [TRT] [I] [MemUsageChange] Init CUDA: CPU +313, GPU +0, now: CPU 406, GPU 1119 (MiB)
[11/23/2022-08:11:53] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 406 MiB, GPU 1119 MiB
[11/23/2022-08:11:53] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 541 MiB, GPU 1153 MiB
[11/23/2022-08:11:53] [TRT] [W] parsers/onnx/onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[11/23/2022-08:11:54] [TRT] [W] parsers/onnx/onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[11/23/2022-08:11:56] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[11/23/2022-08:11:56] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[11/23/2022-08:11:56] [TRT] [I] Successfully created plugin: TRTBatchedNMS
[11/23/2022-08:11:57] [TRT] [W] TensorRT was linked against cuBLAS/cuBLASLt 11.6.5 but loaded cuBLAS/cuBLASLt 11.5.1
[11/23/2022-08:11:57] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +483, GPU +206, now: CPU 1499, GPU 1359 (MiB)
[11/23/2022-08:11:57] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +115, GPU +52, now: CPU 1614, GPU 1411 (MiB)
[11/23/2022-08:11:57] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[11/23/2022-08:14:19] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[11/23/2022-08:19:48] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[11/23/2022-08:19:48] [TRT] [W]  (# 3 (SHAPE input))
[11/23/2022-08:19:48] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:19:48] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:19:48] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:19:48] [TRT] [W]  (# 2 (SHAPE input))
[11/23/2022-08:19:48] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:19:51] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[11/23/2022-08:19:51] [TRT] [W]  (# 3 (SHAPE input))
[11/23/2022-08:19:51] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:19:51] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:19:51] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:19:51] [TRT] [W]  (# 2 (SHAPE input))
[11/23/2022-08:19:51] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:20:21] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[11/23/2022-08:20:21] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:20:21] [TRT] [W]  (# 3 (SHAPE input))
[11/23/2022-08:20:21] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:20:21] [TRT] [W]  (# 2 (SHAPE input))
[11/23/2022-08:20:21] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:20:21] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:20:24] [TRT] [I] Detected 1 inputs and 2 output network tensors.
[11/23/2022-08:20:25] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[11/23/2022-08:20:25] [TRT] [W]  (# 3 (SHAPE input))
[11/23/2022-08:20:25] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:20:25] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:20:25] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:20:25] [TRT] [W]  (# 2 (SHAPE input))
[11/23/2022-08:20:25] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:20:25] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[11/23/2022-08:20:25] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:20:25] [TRT] [W]  (# 3 (SHAPE input))
[11/23/2022-08:20:25] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[11/23/2022-08:20:25] [TRT] [W]  (# 2 (SHAPE input))
[11/23/2022-08:20:25] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:20:25] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[11/23/2022-08:20:25] [TRT] [I] Total Host Persistent Memory: 148064
[11/23/2022-08:20:25] [TRT] [I] Total Device Persistent Memory: 112251904
[11/23/2022-08:20:25] [TRT] [I] Total Scratch Memory: 177525632
[11/23/2022-08:20:25] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 118 MiB, GPU 823 MiB
[11/23/2022-08:20:25] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 214.546ms to assign 22 blocks to 227 nodes requiring 215782400 bytes.
[11/23/2022-08:20:25] [TRT] [I] Total Activation Memory: 215782400
[11/23/2022-08:20:25] [TRT] [W] TensorRT was linked against cuBLAS/cuBLASLt 11.6.5 but loaded cuBLAS/cuBLASLt 11.5.1
[11/23/2022-08:20:25] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 2276, GPU 1829 (MiB)
[11/23/2022-08:20:25] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 2276, GPU 1837 (MiB)
[11/23/2022-08:20:25] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +117, GPU +178, now: CPU 117, GPU 178 (MiB)
2022-11-23 08:20:25,974 - mmdeploy - INFO - Finish pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt
2022-11-23 08:20:26,852 - mmdeploy - INFO - visualize tensorrt model start.
2022-11-23 08:20:30,640 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
2022-11-23:08:20:30 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
2022-11-23 08:20:30,641 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
2022-11-23:08:20:30 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
[11/23/2022-08:20:32] [TRT] [W] TensorRT was linked against cuBLAS/cuBLASLt 11.6.5 but loaded cuBLAS/cuBLASLt 11.5.1
[11/23/2022-08:20:32] [TRT] [W] TensorRT was linked against cuBLAS/cuBLASLt 11.6.5 but loaded cuBLAS/cuBLASLt 11.5.1
/opt/conda/lib/python3.8/site-packages/mmdet/datasets/utils.py:66: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  warnings.warn(
2022-11-23 08:20:35,753 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2022-11-23:08:20:35 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2022-11-23 08:20:36,430 - mmdeploy - INFO - visualize tensorrt model success.
2022-11-23 08:20:36,430 - mmdeploy - INFO - visualize pytorch model start.
load checkpoint from local path: ../v_data/yolov3_dep/epoch_273.pth
/opt/conda/lib/python3.8/site-packages/mmdet/datasets/utils.py:66: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  warnings.warn(
2022-11-23 08:20:57,669 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2022-11-23:08:20:57 - mmdeploy - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
2022-11-23 08:20:58,206 - mmdeploy - INFO - visualize pytorch model success.
2022-11-23 08:20:58,207 - mmdeploy - INFO - All process success.

so i tried to load onnx file,the output and codes as follows:

import onnx
import onnxruntime
import cv2
import numpy as np
input_img = cv2.imread('image.png').astype(np.float32)

# HWC to NCHW
input_img = np.transpose(input_img, [2, 0, 1])
input_img = np.expand_dims(input_img, 0)
ort_session = onnxruntime.InferenceSession("/root/workspace/mmdeploy/work_dir/end2end.onnx")

Error traceback

Traceback (most recent call last):
  File "2.py", line 12, in <module>
    ort_session = onnxruntime.InferenceSession("/root/workspace/mmdeploy/work_dir/end2end.onnx")
  File "/opt/conda/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 283, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/opt/conda/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 310, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /root/workspace/mmdeploy/work_dir/end2end.onnx failed:Fatal error: TRTBatchedNMS is not a registered function/op

Answered by lvhan028

Nov 24, 2022

The onnx model end2end.onnx includes customized onnx operators, which are specially designed for converting to tensorrt model.
It cannot be inferred by onnxruntime.

View full answer

lvhan028 · 2022-11-24T13:19:30Z

lvhan028
Nov 24, 2022
Maintainer

The onnx model end2end.onnx includes customized onnx operators, which are specially designed for converting to tensorrt model.
It cannot be inferred by onnxruntime.

1 reply

emiya777 Nov 25, 2022
Author

oh thx~

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

deploy the yolov3 model to TensorRT #1422

{{title}}

Replies: 1 comment 1 reply

{{title}}

{{title}}

Select a reply

deploy the yolov3 model to TensorRT #1422

emiya777 Nov 23, 2022

Replies: 1 comment · 1 reply

lvhan028 Nov 24, 2022 Maintainer

emiya777 Nov 25, 2022 Author

emiya777
Nov 23, 2022

Replies: 1 comment 1 reply

lvhan028
Nov 24, 2022
Maintainer

emiya777 Nov 25, 2022
Author