Skip to content

Cover caller-stream TLS semantics #21291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shoumikhin wants to merge 1 commit into
base: main
Choose a base branch
from
+25 −0
Open

Cover caller-stream TLS semantics #21291

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions backends/aoti/slim/cuda/test/test_cuda_stream_guard.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
#include <executorch/runtime/platform/platform.h>
#include <gtest/gtest.h>

#include <thread>
#include <type_traits>

using namespace executorch::backends::cuda;
Expand Down Expand Up @@ -296,6 +297,30 @@ TEST(CallerStreamGuardTest, GuardSelectsThenRestores) {
EXPECT_FALSE(getCallerStream().has_value());
}

TEST(CallerStreamGuardTest, ExplicitNullStreamIsStillSelected) {
CallerStreamGuard guard(nullptr);
ASSERT_TRUE(getCallerStream().has_value());
EXPECT_EQ(*getCallerStream(), nullptr);
}

TEST(CallerStreamGuardTest, SelectionIsThreadLocal) {
const cudaStream_t selected = fake_stream(0);
CallerStreamGuard guard(selected);

bool child_started_without_stream = false;
bool child_selected_own_stream = false;
std::thread child([&]() {

@Gasoonjia Gasoonjia Jul 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious why should be take care of the multi-cpu-thread scenerio? Is that something ET need to support?

@shoumikhin shoumikhin Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not about multi-GPU or concurrent execution of one Method. getCallerStream()/CallerStreamGuard are thread_local by design so the CUDA and TRT delegates can each run on their own thread and stream. This is the only test that catches caller_stream_ accidentally losing thread_local; without it, concurrent inferences would silently stomp each other's stream. Keeping it.

child_started_without_stream = !getCallerStream().has_value();
CallerStreamGuard child_guard(fake_stream(1));
child_selected_own_stream = getCallerStream() == fake_stream(1);
});
child.join();

EXPECT_TRUE(child_started_without_stream);
EXPECT_TRUE(child_selected_own_stream);
EXPECT_EQ(getCallerStream(), selected);
Comment thread
shoumikhin marked this conversation as resolved.
}

TEST(CallerStreamGuardTest, NestedGuardsRestoreOuter) {
const cudaStream_t outer = fake_stream(1);
const cudaStream_t inner = fake_stream(2);
Expand Down
Loading