add runtime_platform_compatibility flag

[lanluo-nvidia]

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[lanluo-nvidia]

when I locally test it I got the following error on https://github.com/pytorch/TensorRT/blob/main/py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py#L150:

torch_tensorrt [TensorRT Conversion Context]:logging.py:24 IRuntime::deserializeCudaEngine: Error Code 1: Serialization (Serialization assertion plan->header.pad == expectedPlatformTag failed.Platform specific tag mismatch detected. TensorRT plan files are only supported on the target runtime platform they were created on.)

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-15 21:14:09.852290+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-15 21:14:29.311129+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

[narendasan]

For this make sure to add the cross_compile flag as one of the engine invariant settings so that the cache doesnt try to pull them by accident

[peri044]

• torch_tensorrt.dynamo.cross_compile(model, inputs, *args, **kwargs) -> save EP to disk by calling torch_tensorrt.save()

[peri044]

Options:

1. setup_engine -> execute_engine (Issue here is type is not supported in torch custom ops. For C++ ops, it is working). So POC with custom C++ ops should unblock this workflow.
2. TRTEngine( check with the flag cross_compatibility)
3. execute_serialized_engine() (downside: can't deserialize on every inference)