Skip to content

Add C++ benchmarks: FileHandle sequential read (part 2/n) #874

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
kingcrimsontianyu wants to merge 44 commits into
base: main
Choose a base branch
from
Draft

Add C++ benchmarks: FileHandle sequential read (part 2/n) #874

kingcrimsontianyu wants to merge 44 commits into from

Conversation

@kingcrimsontianyu
Copy link
Contributor

@kingcrimsontianyu kingcrimsontianyu commented Nov 19, 2025 •
edited
Loading

Partially addresses #606

@copy-pr-bot
Copy link

copy-pr-bot bot commented Nov 19, 2025

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@kingcrimsontianyu kingcrimsontianyu added breaking Introduces a breaking change c++ Affects the C++ API of KvikIO feature request New feature or request labels Nov 19, 2025
@copy-pr-bot
Copy link

copy-pr-bot bot commented Nov 21, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@kingcrimsontianyu kingcrimsontianyu mentioned this pull request Nov 24, 2025
Merged
rapids-bot bot pushed a commit that referenced this pull request Nov 26, 2025
@kingcrimsontianyu
Add thread pool as function parameters to C++ API (#876)
2481f31 
This PR introduces two changes in KvikIO C++ API:
- For the `pread` (and `pwrite` if applicable) method in `FileHandle`, `RemoteHandle` and `MmapHandle`'s , this PR adds the thread pool as a function parameter. By default, the global thread pool is used.
- After previous PR's cleanup (https://github.com/rapidsai/kvikio/pull/851/files/30543a3ccb953b0eb2afb7bb91d36ceda482dd69#r2432486309), the `thread_pool_wrapper` class merely forwards calls without adding useful functionality. This PR removes it and adds a simple type alias `ThreadPool` for the underlying `BS:thread_pool`.

This PR is a dependency of #874 which facilitates investigation into a multi-drive scaling problem reported by #850.

Authors:
  - Tianyu Liu (https://github.com/kingcrimsontianyu)

Approvers:
  - Mads R. B. Kristensen (https://github.com/madsbk)

URL: #876
@kingcrimsontianyu
Copy link
Contributor Author

kingcrimsontianyu commented Dec 10, 2025

How to use

  • To build the C++ benchmark in the devcontainer:
build-kvikio-cpp -DKvikIO_BUILD_TESTS=ON -DKvikIO_BUILD_BENCHMARKS=ON -j 0
  • To run the C++ benchmark in the devcontainer:
    • Mount external drives
    • Run the following command. Note that --threads 4 have different meanings according to KVIKIO_THREAD_POOL_PER_BLOCK_DEVICE:
      • =1: 4 threads per hard drive
      • =0: 4 threads for the default global thread pool, shared by all drives
/home/coder/kvikio/cpp/build/latest/benchmarks/POSIX_BENCHMARK --file /mnt/nvme1/file1.bin --file /mnt/nvme2/file2.bin --threads 4

@kingcrimsontianyu kingcrimsontianyu mentioned this pull request Dec 10, 2025
Closed
@kingcrimsontianyu kingcrimsontianyu added non-breaking Introduces a non-breaking change and removed breaking Introduces a breaking change labels Dec 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

c++ Affects the C++ API of KvikIO feature request New feature or request non-breaking Introduces a non-breaking change

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@kingcrimsontianyu