[Build] libonnxruntime_providers_cuda.so with error: libcublasLt.so.12: cannot open shared object file: No such file or directory

[wuhongsheng]

Describe the issue

C++ infer output this error

Urgency

No response

Target platform

linux/amd64

Build script

ScopedTimer timer("Overall detection");

float* blobPtr = nullptr; // Pointer to hold preprocessed image data
// Define the shape of the input tensor (batch size, channels, height, width)
std::vector<int64_t> inputTensorShape = {1, 3, inputImageShape.height, inputImageShape.width};

// Preprocess the image and obtain a pointer to the blob
cv::Mat preprocessedImage = preprocess(image, blobPtr, inputTensorShape);

// Compute the total number of elements in the input tensor
size_t inputTensorSize = utils::vectorProduct(inputTensorShape);

// Create a vector from the blob data for ONNX Runtime input
std::vector<float> inputTensorValues(blobPtr, blobPtr + inputTensorSize);

delete[] blobPtr; // Free the allocated memory for the blob

// Create an Ort memory info object (can be cached if used repeatedly)
static Ort::MemoryInfo memoryInfo = Ort::MemoryInfo::CreateCpu(OrtArenaAllocator, OrtMemTypeDefault);
// Create input tensor object using the preprocessed data
Ort::Value inputTensor = Ort::Value::CreateTensor<float>(
    memoryInfo,
    inputTensorValues.data(),
    inputTensorSize,
    inputTensorShape.data(),
    inputTensorShape.size()
);

// Run the inference session with the input tensor and retrieve output tensors
std::vector<Ort::Value> outputTensors = session.Run(
    Ort::RunOptions{nullptr},
    inputNames.data(),
    &inputTensor,
    numInputNodes,
    outputNames.data(),
    numOutputNodes
);

// Determine the resized image shape based on input tensor shape
cv::Size resizedImageShape(static_cast<int>(inputTensorShape[3]), static_cast<int>(inputTensorShape[2]));

// Postprocess the output tensors to obtain detections
std::vector<Detection> detections = postprocess(image.size(), resizedImageShape, outputTensors, confThreshold, iouThreshold);

return detections; // Return the vector of detections

Error / output

Build files have been written to: /home/zhibo/whs/yolos-cpp/build
root@zhibo:/home/zhibo/whs/yolos-cpp/build# make -j4
[ 16%] Building CXX object CMakeFiles/image_inference.dir/src/image_inference.cpp.o
[ 33%] Building CXX object CMakeFiles/camera_inference.dir/src/camera_inference.cpp.o
[ 50%] Building CXX object CMakeFiles/video_inference.dir/src/video_inference.cpp.o
[ 66%] Linking CXX executable camera_inference
[ 83%] Linking CXX executable image_inference
[100%] Linking CXX executable video_inference
[100%] Built target image_inference
[100%] Built target camera_inference
[100%] Built target video_inference
root@zhibo:/home/zhibo/whs/yolos-cpp/build# ./video_inference
Inference device: GPU
terminate called after throwing an instance of 'Ort::Exception'
what(): /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1539 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.12: cannot open shared object file: No such file or directory

Visual Studio Version

No response

GCC / Compiler Version

No response

[wuhongsheng]

root@zhibo:/home/zhibo/whs/yolos-cpp/build# ls /usr/local/cuda/lib64/ |grep libcublasLt.so.12
libcublasLt.so.12
libcublasLt.so.12.6.4.1

[skottmckay]

Is /usr/local/cuda/lib64 in the PATH when you're running ORT and do the permissions allow ORT to load it?

Are there any dependencies of libcublasLt.so.12 that are not satisfied?

[wejoncy]

Can set LD_LIBRARY_PATH=/usr/local/cuda/lib64/ tempory to test if it works