make aoti_torch_empty_strided support creating incontiguous tensor #15228

Gasoonjia · 2025-10-17T18:47:22Z

Stack from ghstack (oldest at bottom):

This diff modifies the aoti_torch_empty_strided function to support the creation of incontiguous tensors.

To achieve it, this diff:

update the way to calculate the memory size by using both tensor size and the stride
skip stride check in ETensor by adding and checking cmake macro USE_CUDA_BACKEND when building with CUDA backend support.

we will soon bring the ETensor check back for every backend after migrating to use slimtensor.

Differential Revision: D84938258

This diff modifies the `aoti_torch_empty_strided` function to support the creation of incontiguous tensors. To achieve it, this diff: 1. update the way to calculate the memory size by using both tensor size and the stride 2. skip stride check in ETensor by adding and checking cmake macro `USE_CUDA_BACKEND` when building with CUDA backend support. we will soon bring the ETensor check back for every backend after migrating to use slimtensor. Differential Revision: [D84938258](https://our.internmc.facebook.com/intern/diff/D84938258/) [ghstack-poisoned]

pytorch-bot · 2025-10-17T18:47:26Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15228

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Unrelated Failures

As of commit 29d0578 with merge base 7349bb5 ():

NEW FAILURE - The following job has failed:

pull / unittest-nxp-neutron / linux-job (gh)
RuntimeError: Command docker exec -t 9337342c9469b278dccd58d7fa7cd025266829b53ac826435526ce5c80c595e1 /exec failed with exit code 1

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / unittest / linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / unittest-editable / linux / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This diff modifies the `aoti_torch_empty_strided` function to support the creation of incontiguous tensors. To achieve it, this diff: 1. update the way to calculate the memory size by using both tensor size and the stride 2. skip stride check in ETensor by adding and checking cmake macro `USE_CUDA_BACKEND` when building with CUDA backend support. we will soon bring the ETensor check back for every backend after migrating to use slimtensor. Differential Revision: [D84938258](https://our.internmc.facebook.com/intern/diff/D84938258/) ghstack-source-id: 317035163 Pull Request resolved: #15228

github-actions · 2025-10-17T18:48:06Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

…s tensor" This diff modifies the `aoti_torch_empty_strided` function to support the creation of incontiguous tensors. To achieve it, this diff: 1. update the way to calculate the memory size by using both tensor size and the stride 2. skip stride check in ETensor by adding and checking cmake macro `USE_CUDA_BACKEND` when building with CUDA backend support. we will soon bring the ETensor check back for every backend after migrating to use slimtensor. Differential Revision: [D84938258](https://our.internmc.facebook.com/intern/diff/D84938258/) [ghstack-poisoned]

Pull Request resolved: #15228 This diff modifies the `aoti_torch_empty_strided` function to support the creation of incontiguous tensors. To achieve it, this diff: 1. update the way to calculate the memory size by using both tensor size and the stride 2. skip stride check in ETensor by adding and checking cmake macro `USE_CUDA_BACKEND` when building with CUDA backend support. we will soon bring the ETensor check back for every backend after migrating to use slimtensor. ghstack-source-id: 317086685 @exported-using-ghexport Differential Revision: [D84938258](https://our.internmc.facebook.com/intern/diff/D84938258/)

msluszniak · 2025-10-20T14:29:15Z

backends/cuda/runtime/shims/memory.cpp

+    int64_t stride_i = (strides_ptr != nullptr) ? strides_ptr[i] : 0;
+    if (strides_ptr == nullptr) {
+      // Calculate contiguous stride if not provided
+      stride_i = 1;


Suggested change

int64_t stride_i = (strides_ptr != nullptr) ? strides_ptr[i] : 0;

if (strides_ptr == nullptr) {

// Calculate contiguous stride if not provided

stride_i = 1;

int64_t stride_i = (strides_ptr != nullptr) ? strides_ptr[i] : 1;

if (strides_ptr == nullptr) {

// Calculate contiguous stride if not provided

…s tensor" This diff modifies the `aoti_torch_empty_strided` function to support the creation of incontiguous tensors. To achieve it, this diff: 1. update the way to calculate the memory size by using both tensor size and the stride 2. skip stride check in ETensor by adding and checking cmake macro `USE_CUDA_BACKEND` when building with CUDA backend support. we will soon bring the ETensor check back for every backend after migrating to use slimtensor. Differential Revision: [D84938258](https://our.internmc.facebook.com/intern/diff/D84938258/) [ghstack-poisoned]

extension/tensor/tensor_ptr.cpp

…s tensor" This diff modifies the `aoti_torch_empty_strided` function to support the creation of incontiguous tensors. To achieve it, this diff: 1. update the way to calculate the memory size by using both tensor size and the stride 2. skip stride check in ETensor by adding and checking cmake macro `USE_CUDA_BACKEND` when building with CUDA backend support. we will soon bring the ETensor check back for every backend after migrating to use slimtensor. Differential Revision: [D84938258](https://our.internmc.facebook.com/intern/diff/D84938258/) [ghstack-poisoned]

Pull Request resolved: #15228 This diff modifies the `aoti_torch_empty_strided` function to support the creation of incontiguous tensors. To achieve it, this diff: 1. update the way to calculate the memory size by using both tensor size and the stride 2. skip stride check in ETensor by adding and checking cmake macro `USE_CUDA_BACKEND` when building with CUDA backend support. we will soon bring the ETensor check back for every backend after migrating to use slimtensor. ghstack-source-id: 317525743 @exported-using-ghexport Differential Revision: [D84938258](https://our.internmc.facebook.com/intern/diff/D84938258/)

larryliu0820 · 2025-10-21T01:22:14Z

extension/tensor/tensor_ptr.cpp

+// Skip stride calculation and incontiguous tensor check for CUDA backend since
+// AOTI-CUDA handles both contiguous and incontiguous tensors. This will be
+// removed after SlimTensor migration.
+#ifndef USE_CUDA_BACKEND


This is not needed anymore?

you are absolutely right. Removed.

…s tensor" This diff modifies the `aoti_torch_empty_strided` function to support the creation of incontiguous tensors. To achieve it, this diff: 1. update the way to calculate the memory size by using both tensor size and the stride 2. skip stride check in ETensor by adding and checking cmake macro `USE_CUDA_BACKEND` when building with CUDA backend support. we will soon bring the ETensor check back for every backend after migrating to use slimtensor. Differential Revision: [D84938258](https://our.internmc.facebook.com/intern/diff/D84938258/) [ghstack-poisoned]

@Gasoonjia

…15321) This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #15228 by @Gasoonjia ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/gasoonjia/59/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/gasoonjia/59/head Merge bot PR base: https://github.com/pytorch/executorch/tree/main Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/gasoonjia/59/orig Differential Revision: [D84938258](https://our.internmc.facebook.com/intern/diff/D84938258/) @diff-train-skip-merge Co-authored-by: gasoonjia <[email protected]> Co-authored-by: Gasoonjia <[email protected]>

Gasoonjia requested review from kirklandsign, larryliu0820 and shoumikhin as code owners October 17, 2025 18:47

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 17, 2025

meta-codesync bot added fb-exported meta-exported labels Oct 17, 2025

Gasoonjia mentioned this pull request Oct 17, 2025

add module level benchmark for gemma3 model #15241

Merged

msluszniak reviewed Oct 20, 2025

View reviewed changes

Gasoonjia mentioned this pull request Oct 20, 2025

gemma3 e2e runner on cuda #15282

Merged

JacobSzwejbka reviewed Oct 20, 2025

View reviewed changes

extension/tensor/tensor_ptr.cpp Outdated Show resolved Hide resolved

Gasoonjia added 2 commits October 20, 2025 17:14

larryliu0820 reviewed Oct 21, 2025

View reviewed changes

Gasoonjia added 3 commits October 20, 2025 21:31

Gasoonjia mentioned this pull request Oct 21, 2025

adding cuda memory estimation support #15294

Draft

larryliu0820 approved these changes Oct 21, 2025

View reviewed changes

meta-codesync bot merged commit d077c0f into gh/gasoonjia/59/base Oct 21, 2025
142 of 147 checks passed

meta-codesync bot deleted the gh/gasoonjia/59/head branch October 21, 2025 20:24

meta-codesync bot temporarily deployed to cherry-pick-bot October 21, 2025 20:24 Inactive

pytorchbot mentioned this pull request Oct 21, 2025

make aoti_torch_empty_strided support creating incontiguous tensor #15321

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

make aoti_torch_empty_strided support creating incontiguous tensor #15228

make aoti_torch_empty_strided support creating incontiguous tensor #15228

Gasoonjia commented Oct 17, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Oct 17, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Oct 17, 2025

Uh oh!

msluszniak Oct 20, 2025

Uh oh!

Uh oh!

larryliu0820 Oct 21, 2025

Uh oh!

Gasoonjia Oct 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

make aoti_torch_empty_strided support creating incontiguous tensor #15228

make aoti_torch_empty_strided support creating incontiguous tensor #15228

Conversation

Gasoonjia commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15228

❌ 1 New Failure, 2 Unrelated Failures

Uh oh!

github-actions bot commented Oct 17, 2025

This PR needs a release notes: label

Uh oh!

msluszniak Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

larryliu0820 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

Gasoonjia Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Gasoonjia commented Oct 17, 2025 •

edited

Loading

pytorch-bot bot commented Oct 17, 2025 •

edited

Loading

This PR needs a `release notes:` label