[Feature Request] ONNX model file decryption/custom I/O hooks

[vadimkantorov]

Describe the feature request

For on-premises scenarios it might be good to allow the user to provide some hooks for reading/decrypting the model file for all main EP's including the CUDA EP.

This could be added by allowing the user to specify the .so shared file with custom I/O functions (Something like this already exists for TRT EP) or e.g. exporting I/O functions and allowing user to override them with LD_PRELOAD'ing custom overrides

• [Documentation] Document TensorRT engine encryption dll specification #22496
• [TRT EP] Fix logic to reach cache encryption code. #17111
• Decrypt a TRT engine file before deserializing it #15508

Maybe also these I/O hooks could be used to implement some weight loading from S3 or from a custom user's checkpoint blob database.

Describe scenario use case

(For on-premises scenarios it might be good to allow the user to provide some hooks for reading/decrypting the model file)

[MaanavD]

Hey @vadimkantorov, wondering more about the goal behind doing this - what's your personal use case? Do you want onnxruntime to support encrypted models?

[vadimkantorov]

Deployment of on-prem model hosted in Triton/ORT with encrypted model ONNX file. Some foolproof decryption like provided for TRT EP engine encrypted cache would do (of course, a dedicated reverse engineer could do a dump of ORT process memory and extract the model weights and model graph)