Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core] Fix std::flush with istream::readsome #50248

Merged
merged 13 commits into from
Feb 12, 2025

Conversation

dentiny
Copy link
Contributor

@dentiny dentiny commented Feb 5, 2025

Current implementation doesn't handle std::flush, which requires to dump content into sinks right away.
This PR reads content from pipe in two ways:

  • Leverage istream::readsome to apply non-blocking read, to read all content right away;
  • Use istream::read to block read, so we don't waste CPU cycle on busy polling when no content in pipe.

@dentiny dentiny added the go add ONLY when ready to merge, run all tests label Feb 5, 2025
@dentiny dentiny requested a review from jjyao February 5, 2025 11:47
@dentiny dentiny force-pushed the hjiang/pipe-readsome branch from d637a3a to bc19684 Compare February 5, 2025 11:51
Signed-off-by: dentiny <[email protected]>
@dentiny dentiny force-pushed the hjiang/pipe-readsome branch from bc19684 to 0953813 Compare February 5, 2025 11:55
src/ray/util/pipe_logger.cc Outdated Show resolved Hide resolved
@dentiny dentiny requested a review from jjyao February 5, 2025 19:34
@dentiny dentiny force-pushed the hjiang/pipe-readsome branch from 681e78f to 059d963 Compare February 8, 2025 06:54
Signed-off-by: dentiny <[email protected]>
@dentiny dentiny force-pushed the hjiang/pipe-readsome branch from 059d963 to f25d458 Compare February 8, 2025 06:54
@dentiny
Copy link
Contributor Author

dentiny commented Feb 10, 2025

@jjyao I updated the implementation and integrate with the newliner sink, no known stuck issue at the moment.

Signed-off-by: dentiny <[email protected]>
Signed-off-by: dentiny <[email protected]>
@dentiny dentiny force-pushed the hjiang/pipe-readsome branch from 4b96c9b to 9d2ebc3 Compare February 10, 2025 21:55
src/ray/util/pipe_logger.cc Outdated Show resolved Hide resolved
src/ray/util/pipe_logger.h Outdated Show resolved Hide resolved
Signed-off-by: dentiny <[email protected]>
Signed-off-by: dentiny <[email protected]>
@dentiny dentiny requested a review from jjyao February 11, 2025 00:01
Copy link
Collaborator

@jjyao jjyao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG

src/ray/util/tests/pipe_logger_test.cc Outdated Show resolved Hide resolved
src/ray/util/tests/spdlog_newliner_sink_test.cc Outdated Show resolved Hide resolved
std::this_thread::sleep_for(std::chrono::seconds(2));
FlushOnRedirectedStderr();

// Make sure flush hook works fine and process terminates with no problem.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are not checking anything here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See documentation above.

// This test case only checks whether stream redirection process could exit normally.

StreamRedirectionOption opts;
opts.file_path = test_file_path;
opts.tee_to_stderr = true;
RedirectStderr(opts);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we decouple RedirectStream with atexit() so that we can test better and don't need to create a file for each test.

Ideally we can manually call SyncOnStreamRedirection to close the redirection.

RedirectStderr();
// test
SyncOnStreamRedirection(); // restore to the old state
// some checks


RedirectStderr();
// another test
SyncOnStreamRedirection();
// some checks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That way you need to return redirection handler to the caller? Or return the exit terminator.

@dentiny dentiny requested a review from jjyao February 11, 2025 06:03
Signed-off-by: dentiny <[email protected]>
@dentiny dentiny force-pushed the hjiang/pipe-readsome branch from 87dee19 to babd0bb Compare February 11, 2025 07:11
@jjyao jjyao merged commit b11c571 into ray-project:master Feb 12, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
go add ONLY when ready to merge, run all tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants