feat(rocthrust): use primbench#8499
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. ❌ Your project status has failed because the head coverage (76.84%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #8499 +/- ##
===========================================
+ Coverage 64.60% 64.61% +0.01%
===========================================
Files 2687 2686 -1
Lines 423273 423184 -89
Branches 62933 62925 -8
===========================================
- Hits 273439 273414 -25
+ Misses 129163 129097 -66
- Partials 20671 20673 +2
*This pull request uses carry forward flags. Click here to find out more. 🚀 New features to boost your workflow:
|
e48fde6 to
7babd0a
Compare
|
I've got a TheRock PR up here to make amd_smi available for rocThrust: ROCm/TheRock#6100. |
|
@MyNameIsTrez, I applied the recommendation to turn |
|
@MyNameIsTrez I have removed all the |
❌ PR Check — Action Required
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🚫 Please fix the failed policies before requesting reviews. The following policy checks failed:
The |
|
In case anyone ever asks in the future btw @NguyenNhuDi, the two gifs in primbench its readme were generated using asciinema + agg. Here are the details from my personal notes on how I produced those gifs, in case they ever need to be updated. Feel free to copy my notes to a more permanent spot, and to modify them: DetailsContextasciinema is one of the most popular command-line tools to record your terminal with. agg (asciinema gif generator) is a command-line tool for generating GIFs from terminal recordings. InstallationYou can add this to your Dockerfile to download asciinema: And you can add this to your Dockerfile to download agg: UsageRun My typical workflow is:
import json
infile = "recording.cast"
outfile = "recording_trimmed.cast"
trim_start_secs = 1.23
with open(infile) as f:
lines = f.readlines()
header = json.loads(lines[0])
events = [json.loads(line) for line in lines[1:]]
# Filter and shift timestamps
events = [e for e in events if e[0] >= trim_start_secs]
events = [[round(e[0] - trim_start_secs, 6), e[1], e[2]] for e in events]
with open(outfile, "w") as f:
json.dump(header, f, ensure_ascii=False)
f.write("\n")
for e in events:
json.dump(e, f, ensure_ascii=False)
f.write("\n") |
|
@NguyenNhuDi I left some review comments on your commits, in case you hadn't seen them. GitHub apparently hides such review comments, where you have to click this chat icon to view them: 
|
|
@MyNameIsTrez thanks for letting me know. Super strange that it won't show unless you click on it |
|
@NguyenNhuDi The chat icon is more visible on the commits page it turns out, so I recommend scrolling from top-to-bottom there to find the other comments I left: 
|
22d2ea2 to
279dc80
Compare
|
@MyNameIsTrez I have squashed the commits into chunks that makes sense. I believe that this PR is now ready to be reviewed. |
|
@NguyenNhuDi Can you update the description to have this "Test Result" header from hipCUB's PR, which summarizes your results? (without any data) I recommend pressing the Edit button on that hipCUB PR's description, to copy its Test Result header into your rocThrust PR's description, where you update it based on your rocThrust methodology and results. 
|
|
@NguyenNhuDi Can you confirm that the updated |
|
@NguyenNhuDi Great work! Seeing this makes me happy: 
|
|
@NguyenNhuDi Make sure to heed this caution from the top of the readme. Nara has finished that PR of hers just now, so if you approve and merge it then you can rebase your own rocThrust PR onto it immediately: Caution This PR must not be merged before the PR feat(rocthrust): allow grafting thrust onto benchmarks on which it is based has been merged. |
|
I'm done reviewing all of the code now. |
|
@MyNameIsTrez I've generated both cold and hot benchmarks using the run_benchmark script in the gitlab folder. In fact thats what I've used this entire time to generate my benchmarks |
|
@MyNameIsTrez I've taken a look at your review! Thanks for that :) |
|
@NguyenNhuDi It seems you didn't remove the extra spaces before the |
aca3631 to
28ba91a
Compare
|
@MyNameIsTrez oops I did forget to change the commit names. I've done that now |
5 participants
Motivation
Caution
This PR must not be merged before the PR feat(rocthrust): allow grafting thrust onto benchmarks on which it is based has been merged.
rocThrust's benchmarks have been updated to use primbench. This makes the benchmark results more accurate than was possible with Google Benchmark.
The diff of
for_each/basic.curepresents what changes have been made to all benchmarks.JIRA ID : AIPRIMS-208
Technical Details
Note
If AMD's GPU cooling setup is too weak or too powerful then
--min-gpu-temp(default 50°C) or--max-gpu-temp(default 60°C) will need to be passed to benchmarks by AMD's CI. I recommend holding off on passing these flags until a benchmark actually times out from not reaching the 50°C-60°C temperature range.Test Plan
The below blocks of commands can be copied directly into a terminal, so you don't need to copy the lines individually.
See the
Test Resultheader in this description for the zip that contains all of the scripts that these reproduction steps use.VS Code dev containers
HIP
Dockerfile+devcontainer.jsonDockerfile:FROM rocm/rocm-terminal:latestdevcontainer.json:{ "build": { "dockerfile": "Dockerfile" }, "name": "hip-minimal", "runArgs": [ "--device=/dev/kfd", "--device=/dev/dri" ] }CUDA
Dockerfile+devcontainer.jsonDockerfile:devcontainer.json:{ "build": { "dockerfile": "Dockerfile" }, "name": "cuda-minimal", "privileged": true, "runArgs": [ "--gpus=all" ] }Reproduction steps
Installing dependencies
sudo apt update && \ sudo apt install -y cmake ninja-buildCloning
Building primbench benchmarks
Setup
HIP
CUDA
Running primbench benchmarks
python3 .gitlab/run_benchmarks.py \ --benchmark-executables-dir=build_primbench/benchmark \ --gpu-architecture=$gfx \ --json-out-dir=results_primbench \ --csv-out-dir=results_primbenchCopying primbench benchmark results from a server
Building develop benchmarks
Setup
HIP
CUDA
Running develop benchmarks
Copying develop benchmark results from a server
Test Result
No meaningful performance regressions and noise timeouts were observed.
I ran all 40 rocThrust benchmarks on a GFX1201 GPU.
Noisy Benchmarks
1 benchmark still has more than 1% noise:
set_operations_difference.settings.noise_tolerance_percent = 2was set to prevent noise timeout (actual noise is around 1.1%).Paul Kliemann is working on a primbench PR that will significantly lower the noise of benchmark specializations, by calculating entropy instead. hipCUB will automatically benefit from it.
Submission Checklist