Description
When I try to export a model to Riva (using nemo2riva==2.14.0), the FastPitch model export doesn't work (meanwhile, the HiFiGAN export works wonderfully).
I've tried various combinations of onnx and onnxruntime, combinations of NeMo and nemo2riva versions, but in the end, nothing works and the following error always pops up:
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: Invalid tensor data type 0.
Steps/Code to reproduce bug
Using NeMo image (1.21.0, 1.22.0, 1.23.0) + installed nemo2riva==2.14.0:
nemo2riva --out $(OUT_RIVA).riva $(NEMO_MODEL) --key tlt_encode --runtime-check
Additional info
The failed results mentioned by the authors of the issue above can be obtained by:
- Using exported .riva FastPitch from nemo2riva without a
runtime_check
- Downgrading onnx_graphsurgeon to version 0.3.27
In such an environment, the build and deploy of rmir will occur, but at the start of the Riva server, there will be either an:
- IR compatibility issue (version 9, not 8)
- OR
-UNAVAILABLE: Internal: onnx runtime error 1: Load model from /data/models/riva-onnx-fastpitch_encoder-test/1/model.onnx failed:Invalid tensor data type 0.
Description
When I try to export a model to Riva (using nemo2riva==2.14.0), the FastPitch model export doesn't work (meanwhile, the HiFiGAN export works wonderfully).
I've tried various combinations of onnx and onnxruntime, combinations of NeMo and nemo2riva versions, but in the end, nothing works and the following error always pops up:
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: Invalid tensor data type 0.Steps/Code to reproduce bug
Using NeMo image (1.21.0, 1.22.0, 1.23.0) + installed nemo2riva==2.14.0:
nemo2riva --out $(OUT_RIVA).riva $(NEMO_MODEL) --key tlt_encode --runtime-checkAdditional info
--runtime-checkyou can get exported .riva model, but it doesn't work by the end (and the model size is huge, also)The failed results mentioned by the authors of the issue above can be obtained by:
runtime_checkIn such an environment, the build and deploy of rmir will occur, but at the start of the Riva server, there will be either an:
-UNAVAILABLE: Internal: onnx runtime error 1: Load model from /data/models/riva-onnx-fastpitch_encoder-test/1/model.onnx failed:Invalid tensor data type 0.