Please help - inference working in Python but getting errors in C++

[gml16]

Hi, I have an Onnx model that I originally converted from RLlib. I manage to infer this model from Python but I'm struggling to make it work in C++, I'm getting an Ort::Exception at memory location error at Ort::ThrowOnError(const OrtApi & ort, OrtStatus * status) Line 17.

Here is the (very simple) model in question.

Here is my working Python code:

import numpy as np
import onnxruntime

def main():
    exported_model_file = "path/to/saved_model.onnx"
    session = onnxruntime.InferenceSession(exported_model_file, None)
    batch_size = 2
    test_data = {
        "learned-0/obs:0": np.random.uniform(0, 1., size=(batch_size, 71)).astype(np.float32),
        "learned-0/prev_rewards:0": np.random.uniform(-1, 1., size=(batch_size,)).astype(np.float32), # np.array([0.], dtype=np.float32),
        "learned-0/prev_actions:0": np.random.uniform(0, 1, size=(batch_size, 3)).astype(np.int) # np.array([[0., 1.0, 1.0]], dtype=np.int),
    }
    result_onnx = session.run(["learned-0/model/fc_out/BiasAdd:0"], test_data)
    print("Result onnx:", result_onnx)

if __name__ == "__main__":
    main()

Here is the C++ code that is not working:

#include <onnxruntime_cxx_api.h>
#include <onnxruntime_c_api.h>
#include <onnxruntime_run_options_config_keys.h>
#include <onnxruntime_session_options_config_keys.h>

int main()
{
    Ort::Env env{ ORT_LOGGING_LEVEL_VERBOSE, "test" };
    Ort::Session session_{ env, L"path\\to\\saved_model.onnx",
        Ort::SessionOptions{nullptr} };

    const char* input_names[] = { "learned-0/obs:0", "learned-0/prev_rewards:0", "learned-0/prev_actions:0" };
    const char* output_names[] = { "learned-0/model/fc_out/BiasAdd:0" };

    const Ort::MemoryInfo* info = new Ort::MemoryInfo("Cpu", OrtDeviceAllocator, 0, OrtMemTypeDefault);

    std::vector<float> input1;
    for (int i = 0; i < 71; i++)
    {
        input1.emplace_back(0);
        input1.emplace_back(0);
    }
    const std::vector<int64_t> dims1 = { 2, 71 };

    std::vector<Ort::Value> input_tensors;
    input_tensors.push_back(Ort::Value::CreateTensor<float>(*info, input1.data(),
        input1.size(), dims1.data(), dims1.size()));

    const std::vector<int64_t> dims2 = { 2 };
    std::vector<float> input2{ 0, 0 };
    input_tensors.push_back(Ort::Value::CreateTensor<float>(*info, input2.data(),
        input2.size(), dims2.data(), dims2.size()));

    const std::vector<int64_t> dims3 = { 2, 3 };
    std::vector<int> input3{ 0, 0, 0 ,0, 0, 0 };

    input_tensors.push_back(Ort::Value::CreateTensor<int>(*info, input3.data(),
        input3.size(), dims3.data(), dims3.size()));

    std::vector<Ort::Value> output_tensors = session_.Run(Ort::RunOptions{ nullptr }, input_names, (const Ort::Value*)input_tensors.data(), 3, output_names, 1);
}

If it helps here are the model inputs (only the first 3 should be necessary to be fed to the model):

{'name': 'learned-0/prev_actions:0', 'type': {'tensorType': {'elemType': 7, 'shape': {'dim': [{'dimParam': 'unk__235'}, {'dimValue': '3'}]}}}}
{'name': 'learned-0/prev_rewards:0', 'type': {'tensorType': {'elemType': 1, 'shape': {'dim': [{'dimParam': 'unk__236'}]}}}}
{'name': 'learned-0/obs:0', 'type': {'tensorType': {'elemType': 1, 'shape': {'dim': [{'dimParam': 'unk__237'}, {'dimValue': '71'}]}}}}
{'name': 'learned-0/is_training:0', 'type': {'tensorType': {'elemType': 9, 'shape': {}}}}
{'name': 'learned-0/timestep:0', 'type': {'tensorType': {'elemType': 7, 'shape': {}}}}
{'name': 'learned-0/is_exploring:0', 'type': {'tensorType': {'elemType': 9, 'shape': {}}}}

and model outputs (I'm only interested in the 'learned-0/model/fc_out/BiasAdd:0' output, I have also tried to run the inference for all outputs but it gave the same error):

{'name': 'learned-0/Exp:0', 'type': {'tensorType': {'elemType': 1, 'shape': {'dim': [{'dimParam': 'unk__238'}]}}}}
{'name': 'learned-0/Reshape:0', 'type': {'tensorType': {'elemType': 1, 'shape': {'dim': [{'dimParam': 'unk__239'}]}}}}
{'name': 'learned-0/cond_2/Merge:0', 'type': {'tensorType': {'elemType': 1, 'shape': {'dim': [{'dimParam': 'unk__240'}]}}}}
{'name': 'learned-0/cond_1/Merge:0', 'type': {'tensorType': {'elemType': 7, 'shape': {'dim': [{'dimParam': 'unk__241'}, {'dimValue': '3'}]}}}}
{'name': 'learned-0/model/fc_out/BiasAdd:0', 'type': {'tensorType': {'elemType': 1, 'shape': {'dim': [{'dimParam': 'unk__242'}, {'dimValue': '6'}]}}}}

I'm particularly confused about this issue as I have inferred both using Python and C++ a different Onnx model before (with different inputs and outputs) and everything just worked fine.

Any help would be greatly appreciated.
Thank you.

[wschin]

Do you have full call stack? Ort::ThrowOnError(const OrtApi & ort, OrtStatus * status) doesn't look like the root cause line. Thank you.