Benchmark analysis: Print all top rows when asked for #2089

bernhardmgruber · 2024-07-26T15:46:00Z

Analyzing the top 10 result of a tuning run like

benchmarks/scripts/analyze.py --top 10 ./cccl_meta_bench.db

prints those nicely. However, sometimes I need more output, e.g. --top 150. Such output is truncated:

cub.bench.transform.babelstream[T{ct}=I8]:
              variant     score      mins     means       maxs
130  tpb_896.alg_4 ()  8.592724  1.333333  8.592724  21.716392
145  tpb_992.alg_4 ()  8.563333  1.333333  8.563333  21.537903
115  tpb_800.alg_4 ()  8.510113  1.333333  8.510113  22.333335
90   tpb_640.alg_4 ()  8.495584  1.333333  8.495584  20.472222
120  tpb_832.alg_4 ()  8.401476  1.333333  8.401476  21.676473
..                ...       ...       ...       ...        ...
137  tpb_960.alg_1 ()  0.575716  0.377841  0.575716   0.705357
132  tpb_928.alg_1 ()  0.566222  0.377841  0.566222   0.705357
117  tpb_832.alg_1 ()  0.551207  0.377841  0.551207   0.705357
122  tpb_864.alg_1 ()  0.539873  0.377841  0.539873   0.699115
112  tpb_800.alg_1 ()  0.523036  0.346354  0.523036   0.705357

[146 rows x 5 columns]

This PR removes the truncation and prints the number of rows that the user asked for.

bernhardmgruber · 2024-07-26T15:47:08Z

benchmarks/scripts/analyze.py

+    with pd.option_context('display.max_rows', None):
+        print(extract_scores(case_dfs).head(N))


I have no idea if this is the best way to achieve this. I dug this up on the web.

github-actions · 2024-07-27T05:04:19Z

🟩 CI finished in 13h 16m: Pass: 100%/421 | Total: 2d 21h | Avg: 9m 55s | Max: 56m 07s | Hits: 96%/525254

🟩 cub: Pass: 100%/131 | Total: 1d 11h | Avg: 16m 16s | Max: 56m 07s | Hits: 98%/111124

🟩 cpu
  🟩 amd64              Pass: 100%/123 | Total:  1d 05h | Avg: 14m 29s | Max: 56m 07s | Hits:  98%/104188
  🟩 arm64              Pass: 100%/8   | Total:  5h 49m | Avg: 43m 42s | Max: 52m 08s | Hits:  97%/6936  
🟩 ctk
  🟩 11.1               Pass: 100%/15  | Total:  1h 02m | Avg:  4m 10s | Max: 11m 38s | Hits:  99%/11792 
  🟩 11.8               Pass: 100%/3   | Total: 13m 28s | Avg:  4m 29s | Max:  4m 48s | Hits:  99%/2601  
  🟩 12.5               Pass: 100%/113 | Total:  1d 10h | Avg: 18m 11s | Max: 56m 07s | Hits:  98%/96731 
🟩 cudacxx
  🟩 ClangCUDA17        Pass: 100%/2   | Total:  7m 19s | Avg:  3m 39s | Max:  3m 47s | Hits: 100%/1436  
  🟩 nvcc11.1           Pass: 100%/15  | Total:  1h 02m | Avg:  4m 10s | Max: 11m 38s | Hits:  99%/11792 
  🟩 nvcc11.8           Pass: 100%/3   | Total: 13m 28s | Avg:  4m 29s | Max:  4m 48s | Hits:  99%/2601  
  🟩 nvcc12.5           Pass: 100%/111 | Total:  1d 10h | Avg: 18m 27s | Max: 56m 07s | Hits:  98%/95295 
🟩 cudacxx_family
  🟩 ClangCUDA          Pass: 100%/2   | Total:  7m 19s | Avg:  3m 39s | Max:  3m 47s | Hits: 100%/1436  
  🟩 nvcc               Pass: 100%/129 | Total:  1d 11h | Avg: 16m 28s | Max: 56m 07s | Hits:  98%/109688
🟩 cxx
  🟩 Clang9             Pass: 100%/6   | Total: 27m 47s | Avg:  4m 37s | Max:  5m 06s | Hits: 100%/4980  
  🟩 Clang10            Pass: 100%/3   | Total: 16m 05s | Avg:  5m 21s | Max:  5m 29s | Hits: 100%/2607  
  🟩 Clang11            Pass: 100%/4   | Total: 17m 31s | Avg:  4m 22s | Max:  4m 30s | Hits: 100%/3476  
  🟩 Clang12            Pass: 100%/4   | Total: 16m 55s | Avg:  4m 13s | Max:  4m 30s | Hits: 100%/3476  
  🟩 Clang13            Pass: 100%/4   | Total: 17m 30s | Avg:  4m 22s | Max:  4m 27s | Hits: 100%/3476  
  🟩 Clang14            Pass: 100%/4   | Total: 18m 12s | Avg:  4m 33s | Max:  4m 42s | Hits: 100%/3476  
  🟩 Clang15            Pass: 100%/4   | Total: 18m 58s | Avg:  4m 44s | Max:  4m 57s | Hits: 100%/3468  
  🟩 Clang16            Pass: 100%/4   | Total: 18m 39s | Avg:  4m 39s | Max:  4m 48s | Hits: 100%/3468  
  🟩 Clang17            Pass: 100%/26  | Total:  9h 54m | Avg: 22m 52s | Max: 49m 51s | Hits:  99%/22244 
  🟩 GCC6               Pass: 100%/2   | Total:  7m 01s | Avg:  3m 30s | Max:  3m 36s | Hits:  99%/1582  
  🟩 GCC7               Pass: 100%/6   | Total: 23m 48s | Avg:  3m 58s | Max:  4m 40s | Hits:  99%/4983  
  🟩 GCC8               Pass: 100%/6   | Total: 23m 32s | Avg:  3m 55s | Max:  4m 39s | Hits:  99%/4983  
  🟩 GCC9               Pass: 100%/6   | Total: 23m 39s | Avg:  3m 56s | Max:  4m 27s | Hits:  99%/4983  
  🟩 GCC10              Pass: 100%/4   | Total: 17m 26s | Avg:  4m 21s | Max:  4m 31s | Hits:  99%/3476  
  🟩 GCC11              Pass: 100%/7   | Total:  1h 21m | Avg: 11m 41s | Max: 55m 21s | Hits:  94%/6069  
  🟩 GCC12              Pass: 100%/4   | Total: 19m 12s | Avg:  4m 48s | Max:  4m 56s | Hits:  99%/3468  
  🟩 GCC13              Pass: 100%/28  | Total: 13h 09m | Avg: 28m 11s | Max: 54m 26s | Hits:  99%/24276 
  🟩 Intel2023.2.0      Pass: 100%/3   | Total:  2h 06m | Avg: 42m 15s | Max: 43m 35s | Hits:  89%/2379  
  🟩 MSVC14.16          Pass: 100%/1   | Total: 11m 38s | Avg: 11m 38s | Max: 11m 38s | Hits:  99%/709   
  🟩 MSVC14.29          Pass: 100%/2   | Total:  1h 44m | Avg: 52m 12s | Max: 52m 16s | Hits:  88%/1418  
  🟩 MSVC14.39          Pass: 100%/3   | Total:  2h 37m | Avg: 52m 20s | Max: 56m 07s | Hits:  88%/2127  
🟩 cxx_family
  🟩 Clang              Pass: 100%/59  | Total: 12h 26m | Avg: 12m 38s | Max: 49m 51s | Hits:  99%/50671 
  🟩 GCC                Pass: 100%/63  | Total: 16h 25m | Avg: 15m 38s | Max: 55m 21s | Hits:  98%/53820 
  🟩 Intel              Pass: 100%/3   | Total:  2h 06m | Avg: 42m 15s | Max: 43m 35s | Hits:  89%/2379  
  🟩 MSVC               Pass: 100%/6   | Total:  4h 33m | Avg: 45m 30s | Max: 56m 07s | Hits:  90%/4254  
🟩 gpu
  🟩 v100               Pass: 100%/131 | Total:  1d 11h | Avg: 16m 16s | Max: 56m 07s | Hits:  98%/111124
🟩 jobs
  🟩 Build              Pass: 100%/99  | Total: 20h 14m | Avg: 12m 16s | Max: 56m 07s | Hits:  98%/83380 
  🟩 DeviceLaunch       Pass: 100%/8   | Total:  4h 11m | Avg: 31m 22s | Max: 51m 28s | Hits:  99%/6936  
  🟩 GraphCapture       Pass: 100%/8   | Total:  2h 41m | Avg: 20m 10s | Max: 35m 39s | Hits:  99%/6936  
  🟩 HostLaunch         Pass: 100%/8   | Total:  3h 36m | Avg: 27m 01s | Max: 43m 20s | Hits:  99%/6936  
  🟩 TestGPU            Pass: 100%/8   | Total:  4h 48m | Avg: 36m 05s | Max: 54m 26s | Hits:  99%/6936  
🟩 sm
  🟩 60;70;80;90        Pass: 100%/3   | Total: 13m 28s | Avg:  4m 29s | Max:  4m 48s | Hits:  99%/2601  
  🟩 90a                Pass: 100%/4   | Total: 14m 52s | Avg:  3m 43s | Max:  4m 00s | Hits:  99%/3468  
🟩 std
  🟩 11                 Pass: 100%/34  | Total:  7h 24m | Avg: 13m 03s | Max: 51m 15s | Hits:  99%/29047 
  🟩 14                 Pass: 100%/37  | Total:  9h 35m | Avg: 15m 33s | Max: 56m 07s | Hits:  98%/31174 
  🟩 17                 Pass: 100%/36  | Total: 11h 04m | Avg: 18m 27s | Max: 55m 21s | Hits:  97%/30392 
  🟩 20                 Pass: 100%/24  | Total:  7h 27m | Avg: 18m 39s | Max: 51m 20s | Hits:  99%/20511

🟩 thrust: Pass: 100%/118 | Total: 17h 53m | Avg: 9m 05s | Max: 52m 18s | Hits: 96%/138912

🟩 cpu
  🟩 amd64              Pass: 100%/110 | Total: 15h 49m | Avg:  8m 37s | Max: 52m 18s | Hits:  96%/129492
  🟩 arm64              Pass: 100%/8   | Total:  2h 03m | Avg: 15m 26s | Max: 19m 14s | Hits:  89%/9420  
🟩 ctk
  🟩 11.1               Pass: 100%/15  | Total: 58m 59s | Avg:  3m 55s | Max: 13m 52s | Hits:  99%/17660 
  🟩 11.8               Pass: 100%/3   | Total: 11m 35s | Avg:  3m 51s | Max:  3m 59s | Hits:  99%/3534  
  🟩 12.5               Pass: 100%/100 | Total: 16h 42m | Avg: 10m 01s | Max: 52m 18s | Hits:  95%/117718
🟩 cudacxx
  🟩 ClangCUDA17        Pass: 100%/2   | Total:  7m 26s | Avg:  3m 43s | Max:  3m 51s | Hits: 100%/2354  
  🟩 nvcc11.1           Pass: 100%/15  | Total: 58m 59s | Avg:  3m 55s | Max: 13m 52s | Hits:  99%/17660 
  🟩 nvcc11.8           Pass: 100%/3   | Total: 11m 35s | Avg:  3m 51s | Max:  3m 59s | Hits:  99%/3534  
  🟩 nvcc12.5           Pass: 100%/98  | Total: 16h 35m | Avg: 10m 09s | Max: 52m 18s | Hits:  95%/115364
🟩 cudacxx_family
  🟩 ClangCUDA          Pass: 100%/2   | Total:  7m 26s | Avg:  3m 43s | Max:  3m 51s | Hits: 100%/2354  
  🟩 nvcc               Pass: 100%/116 | Total: 17h 45m | Avg:  9m 11s | Max: 52m 18s | Hits:  96%/136558
🟩 cxx
  🟩 Clang9             Pass: 100%/6   | Total: 22m 41s | Avg:  3m 46s | Max:  4m 25s | Hits: 100%/7062  
  🟩 Clang10            Pass: 100%/3   | Total: 13m 32s | Avg:  4m 30s | Max:  4m 53s | Hits: 100%/3531  
  🟩 Clang11            Pass: 100%/4   | Total: 14m 49s | Avg:  3m 42s | Max:  3m 51s | Hits: 100%/4708  
  🟩 Clang12            Pass: 100%/4   | Total: 14m 49s | Avg:  3m 42s | Max:  3m 51s | Hits: 100%/4708  
  🟩 Clang13            Pass: 100%/4   | Total: 15m 20s | Avg:  3m 50s | Max:  4m 11s | Hits: 100%/4708  
  🟩 Clang14            Pass: 100%/4   | Total: 15m 24s | Avg:  3m 51s | Max:  4m 08s | Hits: 100%/4708  
  🟩 Clang15            Pass: 100%/4   | Total: 14m 51s | Avg:  3m 42s | Max:  3m 55s | Hits: 100%/4708  
  🟩 Clang16            Pass: 100%/4   | Total: 15m 38s | Avg:  3m 54s | Max:  3m 57s | Hits: 100%/4708  
  🟩 Clang17            Pass: 100%/18  | Total:  3h 02m | Avg: 10m 07s | Max: 31m 32s | Hits:  98%/21186 
  🟩 GCC6               Pass: 100%/2   | Total:  6m 36s | Avg:  3m 18s | Max:  3m 20s | Hits:  99%/2354  
  🟩 GCC7               Pass: 100%/6   | Total: 19m 48s | Avg:  3m 18s | Max:  3m 41s | Hits:  99%/7068  
  🟩 GCC8               Pass: 100%/6   | Total: 20m 15s | Avg:  3m 22s | Max:  4m 03s | Hits:  99%/7068  
  🟩 GCC9               Pass: 100%/6   | Total: 21m 31s | Avg:  3m 35s | Max:  3m 55s | Hits:  99%/7068  
  🟩 GCC10              Pass: 100%/4   | Total: 14m 33s | Avg:  3m 38s | Max:  3m 49s | Hits:  99%/4712  
  🟩 GCC11              Pass: 100%/7   | Total: 31m 35s | Avg:  4m 30s | Max:  8m 21s | Hits:  93%/8246  
  🟩 GCC12              Pass: 100%/4   | Total: 45m 20s | Avg: 11m 20s | Max: 33m 20s | Hits:  87%/4712  
  🟩 GCC13              Pass: 100%/20  | Total:  3h 18m | Avg:  9m 54s | Max: 20m 14s | Hits:  97%/23560 
  🟩 Intel2023.2.0      Pass: 100%/3   | Total:  1h 34m | Avg: 31m 32s | Max: 34m 14s | Hits:  72%/3540  
  🟩 MSVC14.16          Pass: 100%/1   | Total: 13m 52s | Avg: 13m 52s | Max: 13m 52s | Hits:  98%/1173  
  🟩 MSVC14.29          Pass: 100%/2   | Total:  1h 39m | Avg: 49m 35s | Max: 52m 18s | Hits:  70%/2346  
  🟩 MSVC14.39          Pass: 100%/6   | Total:  3h 18m | Avg: 33m 00s | Max: 48m 35s | Hits:  84%/7038  
🟩 cxx_family
  🟩 Clang              Pass: 100%/51  | Total:  5h 09m | Avg:  6m 04s | Max: 31m 32s | Hits:  99%/60027 
  🟩 GCC                Pass: 100%/55  | Total:  5h 57m | Avg:  6m 30s | Max: 33m 20s | Hits:  97%/64788 
  🟩 Intel              Pass: 100%/3   | Total:  1h 34m | Avg: 31m 32s | Max: 34m 14s | Hits:  72%/3540  
  🟩 MSVC               Pass: 100%/9   | Total:  5h 11m | Avg: 34m 33s | Max: 52m 18s | Hits:  83%/10557 
🟩 gpu
  🟩 v100               Pass: 100%/118 | Total: 17h 53m | Avg:  9m 05s | Max: 52m 18s | Hits:  96%/138912
🟩 jobs
  🟩 Build              Pass: 100%/99  | Total: 13h 33m | Avg:  8m 13s | Max: 52m 18s | Hits:  95%/116553
  🟩 TestCPU            Pass: 100%/11  | Total:  1h 42m | Avg:  9m 18s | Max: 19m 10s | Hits:  99%/12939 
  🟩 TestGPU            Pass: 100%/8   | Total:  2h 37m | Avg: 19m 37s | Max: 31m 32s | Hits:  99%/9420  
🟩 sm
  🟩 60;70;80;90        Pass: 100%/3   | Total: 11m 35s | Avg:  3m 51s | Max:  3m 59s | Hits:  99%/3534  
  🟩 90a                Pass: 100%/4   | Total: 13m 07s | Avg:  3m 16s | Max:  3m 21s | Hits:  99%/4712  
🟩 std
  🟩 11                 Pass: 100%/30  | Total:  2h 55m | Avg:  5m 51s | Max: 27m 54s | Hits:  98%/35328 
  🟩 14                 Pass: 100%/34  | Total:  5h 40m | Avg: 10m 00s | Max: 48m 35s | Hits:  96%/40020 
  🟩 17                 Pass: 100%/33  | Total:  5h 23m | Avg:  9m 47s | Max: 52m 18s | Hits:  96%/38847 
  🟩 20                 Pass: 100%/21  | Total:  3h 53m | Avg: 11m 08s | Max: 47m 56s | Hits:  92%/24717

🟩 libcudacxx: Pass: 100%/112 | Total: 13h 27m | Avg: 7m 12s | Max: 32m 48s | Hits: 95%/273250

🟩 cpu
  🟩 amd64              Pass: 100%/104 | Total: 12h 41m | Avg:  7m 19s | Max: 32m 48s | Hits:  95%/250904
  🟩 arm64              Pass: 100%/8   | Total: 45m 44s | Avg:  5m 43s | Max: 19m 41s | Hits:  91%/22346 
🟩 ctk
  🟩 11.1               Pass: 100%/15  | Total: 55m 10s | Avg:  3m 40s | Max: 16m 15s | Hits:  99%/39780 
  🟩 11.8               Pass: 100%/3   | Total: 41m 40s | Avg: 13m 53s | Max: 19m 53s | Hits:  65%/8064  
  🟩 12.5               Pass: 100%/94  | Total: 11h 50m | Avg:  7m 33s | Max: 32m 48s | Hits:  95%/225406
🟩 cudacxx
  🟩 ClangCUDA17        Pass: 100%/2   | Total: 35m 35s | Avg: 17m 47s | Max: 18m 47s | Hits:  37%/6099  
  🟩 nvcc11.1           Pass: 100%/15  | Total: 55m 10s | Avg:  3m 40s | Max: 16m 15s | Hits:  99%/39780 
  🟩 nvcc11.8           Pass: 100%/3   | Total: 41m 40s | Avg: 13m 53s | Max: 19m 53s | Hits:  65%/8064  
  🟩 nvcc12.5           Pass: 100%/92  | Total: 11h 14m | Avg:  7m 20s | Max: 32m 48s | Hits:  96%/219307
🟩 cudacxx_family
  🟩 ClangCUDA          Pass: 100%/2   | Total: 35m 35s | Avg: 17m 47s | Max: 18m 47s | Hits:  37%/6099  
  🟩 nvcc               Pass: 100%/110 | Total: 12h 51m | Avg:  7m 00s | Max: 32m 48s | Hits:  96%/267151
🟩 cxx
  🟩 Clang9             Pass: 100%/6   | Total: 25m 09s | Avg:  4m 11s | Max:  5m 12s | Hits:  98%/16160 
  🟩 Clang10            Pass: 100%/3   | Total: 15m 11s | Avg:  5m 03s | Max:  5m 14s | Hits:  99%/8109  
  🟩 Clang11            Pass: 100%/4   | Total: 16m 35s | Avg:  4m 08s | Max:  4m 38s | Hits:  98%/11181 
  🟩 Clang12            Pass: 100%/4   | Total: 15m 36s | Avg:  3m 54s | Max:  4m 04s | Hits:  99%/11181 
  🟩 Clang13            Pass: 100%/4   | Total: 16m 12s | Avg:  4m 03s | Max:  4m 55s | Hits:  98%/11181 
  🟩 Clang14            Pass: 100%/4   | Total: 16m 30s | Avg:  4m 07s | Max:  4m 31s | Hits:  99%/11181 
  🟩 Clang15            Pass: 100%/4   | Total: 15m 31s | Avg:  3m 52s | Max:  4m 19s | Hits:  99%/11173 
  🟩 Clang16            Pass: 100%/4   | Total: 16m 07s | Avg:  4m 01s | Max:  4m 24s | Hits:  99%/11173 
  🟩 Clang17            Pass: 100%/14  | Total:  2h 24m | Avg: 10m 17s | Max: 25m 47s | Hits:  85%/28445 
  🟩 GCC6               Pass: 100%/2   | Total:  5m 12s | Avg:  2m 36s | Max:  2m 44s | Hits:  99%/5045  
  🟩 GCC7               Pass: 100%/6   | Total: 34m 15s | Avg:  5m 42s | Max: 19m 38s | Hits:  93%/16146 
  🟩 GCC8               Pass: 100%/6   | Total: 18m 04s | Avg:  3m 00s | Max:  3m 46s | Hits:  99%/16154 
  🟩 GCC9               Pass: 100%/6   | Total: 18m 25s | Avg:  3m 04s | Max:  3m 36s | Hits:  99%/16158 
  🟩 GCC10              Pass: 100%/4   | Total: 13m 39s | Avg:  3m 24s | Max:  3m 54s | Hits:  99%/11181 
  🟩 GCC11              Pass: 100%/7   | Total: 56m 49s | Avg:  8m 07s | Max: 19m 53s | Hits:  84%/19237 
  🟩 GCC12              Pass: 100%/4   | Total: 14m 17s | Avg:  3m 34s | Max:  3m 49s | Hits:  99%/11173 
  🟩 GCC13              Pass: 100%/21  | Total:  4h 19m | Avg: 12m 20s | Max: 32m 48s | Hits:  94%/33902 
  🟩 Intel2023.2.0      Pass: 100%/3   | Total: 28m 56s | Avg:  9m 38s | Max: 18m 45s | Hits:  79%/8099  
  🟩 MSVC14.16          Pass: 100%/1   | Total: 16m 15s | Avg: 16m 15s | Max: 16m 15s | Hits:  99%/2536  
  🟩 MSVC14.29          Pass: 100%/2   | Total: 23m 46s | Avg: 11m 53s | Max: 12m 11s | Hits:  97%/5434  
  🟩 MSVC14.39          Pass: 100%/3   | Total: 37m 35s | Avg: 12m 31s | Max: 13m 31s | Hits:  98%/8401  
🟩 cxx_family
  🟩 Clang              Pass: 100%/47  | Total:  4h 40m | Avg:  5m 58s | Max: 25m 47s | Hits:  95%/119784
  🟩 GCC                Pass: 100%/56  | Total:  6h 59m | Avg:  7m 29s | Max: 32m 48s | Hits:  94%/128996
  🟩 Intel              Pass: 100%/3   | Total: 28m 56s | Avg:  9m 38s | Max: 18m 45s | Hits:  79%/8099  
  🟩 MSVC               Pass: 100%/6   | Total:  1h 17m | Avg: 12m 56s | Max: 16m 15s | Hits:  98%/16371 
🟩 gpu
  🟩 v100               Pass: 100%/112 | Total: 13h 27m | Avg:  7m 12s | Max: 32m 48s | Hits:  95%/273250
🟩 jobs
  🟩 Build              Pass: 100%/99  | Total:  8h 51m | Avg:  5m 22s | Max: 19m 53s | Hits:  95%/273230
  🟩 NVRTC              Pass: 100%/4   | Total:  2h 01m | Avg: 30m 22s | Max: 32m 48s | Hits: 100%/20    
  🟩 Test               Pass: 100%/8   | Total:  2h 32m | Avg: 19m 02s | Max: 25m 47s
  🟩 VerifyCodegen      Pass: 100%/1   | Total:  1m 56s | Avg:  1m 56s | Max:  1m 56s
🟩 sm
  🟩 60;70;80;90        Pass: 100%/3   | Total: 41m 40s | Avg: 13m 53s | Max: 19m 53s | Hits:  65%/8064  
  🟩 90a                Pass: 100%/4   | Total: 14m 23s | Avg:  3m 35s | Max:  4m 00s | Hits:  99%/11536 
🟩 std
  🟩 11                 Pass: 100%/29  | Total:  2h 48m | Avg:  5m 49s | Max: 26m 13s | Hits:  96%/58200 
  🟩 14                 Pass: 100%/32  | Total:  3h 36m | Avg:  6m 45s | Max: 32m 30s | Hits:  96%/81788 
  🟩 17                 Pass: 100%/31  | Total:  3h 51m | Avg:  7m 28s | Max: 29m 59s | Hits:  94%/84134 
  🟩 20                 Pass: 100%/19  | Total:  3h 08m | Avg:  9m 55s | Max: 32m 48s | Hits:  91%/49128

🟩 cudax: Pass: 100%/55 | Total: 2h 17m | Avg: 2m 30s | Max: 7m 19s | Hits: 96%/1968

🟩 cpu
  🟩 amd64              Pass: 100%/51  | Total:  2h 09m | Avg:  2m 31s | Max:  7m 19s | Hits:  96%/1824  
  🟩 arm64              Pass: 100%/4   | Total:  8m 37s | Avg:  2m 09s | Max:  2m 39s | Hits:  97%/144   
🟩 ctk
  🟩 12.0               Pass: 100%/23  | Total: 58m 28s | Avg:  2m 32s | Max:  6m 04s | Hits:  96%/822   
  🟩 12.5               Pass: 100%/32  | Total:  1h 19m | Avg:  2m 28s | Max:  7m 19s | Hits:  97%/1146  
🟩 cudacxx
  🟩 nvcc12.0           Pass: 100%/23  | Total: 58m 28s | Avg:  2m 32s | Max:  6m 04s | Hits:  96%/822   
  🟩 nvcc12.5           Pass: 100%/32  | Total:  1h 19m | Avg:  2m 28s | Max:  7m 19s | Hits:  97%/1146  
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/55  | Total:  2h 17m | Avg:  2m 30s | Max:  7m 19s | Hits:  96%/1968  
🟩 cxx
  🟩 Clang9             Pass: 100%/2   | Total:  4m 24s | Avg:  2m 12s | Max:  2m 15s | Hits: 100%/72    
  🟩 Clang10            Pass: 100%/2   | Total:  4m 18s | Avg:  2m 09s | Max:  2m 09s | Hits: 100%/72    
  🟩 Clang11            Pass: 100%/4   | Total:  8m 16s | Avg:  2m 04s | Max:  2m 07s | Hits: 100%/144   
  🟩 Clang12            Pass: 100%/4   | Total:  8m 14s | Avg:  2m 03s | Max:  2m 10s | Hits: 100%/144   
  🟩 Clang13            Pass: 100%/4   | Total:  8m 05s | Avg:  2m 01s | Max:  2m 05s | Hits: 100%/144   
  🟩 Clang14            Pass: 100%/6   | Total: 16m 22s | Avg:  2m 43s | Max:  4m 14s | Hits: 100%/216   
  🟩 Clang15            Pass: 100%/2   | Total:  4m 07s | Avg:  2m 03s | Max:  2m 04s | Hits: 100%/72    
  🟩 Clang16            Pass: 100%/6   | Total: 17m 43s | Avg:  2m 57s | Max:  4m 22s | Hits: 100%/216   
  🟩 GCC9               Pass: 100%/2   | Total:  3m 52s | Avg:  1m 56s | Max:  1m 56s | Hits:  94%/72    
  🟩 GCC10              Pass: 100%/4   | Total:  7m 41s | Avg:  1m 55s | Max:  2m 03s | Hits:  94%/144   
  🟩 GCC11              Pass: 100%/4   | Total:  7m 37s | Avg:  1m 54s | Max:  2m 00s | Hits:  94%/144   
  🟩 GCC12              Pass: 100%/12  | Total: 31m 15s | Avg:  2m 36s | Max:  4m 07s | Hits:  94%/432   
  🟩 Intel2023.2.0      Pass: 100%/1   | Total:  2m 30s | Avg:  2m 30s | Max:  2m 30s | Hits: 100%/36    
  🟩 MSVC14.36          Pass: 100%/1   | Total:  6m 04s | Avg:  6m 04s | Max:  6m 04s | Hits:  73%/30    
  🟩 MSVC14.39          Pass: 100%/1   | Total:  7m 19s | Avg:  7m 19s | Max:  7m 19s | Hits:  73%/30    
🟩 cxx_family
  🟩 Clang              Pass: 100%/30  | Total:  1h 11m | Avg:  2m 22s | Max:  4m 22s | Hits: 100%/1080  
  🟩 GCC                Pass: 100%/22  | Total: 50m 25s | Avg:  2m 17s | Max:  4m 07s | Hits:  94%/792   
  🟩 Intel              Pass: 100%/1   | Total:  2m 30s | Avg:  2m 30s | Max:  2m 30s | Hits: 100%/36    
  🟩 MSVC               Pass: 100%/2   | Total: 13m 23s | Avg:  6m 41s | Max:  7m 19s | Hits:  73%/60    
🟩 gpu
  🟩 v100               Pass: 100%/55  | Total:  2h 17m | Avg:  2m 30s | Max:  7m 19s | Hits:  96%/1968  
🟩 jobs
  🟩 Build              Pass: 100%/47  | Total:  1h 45m | Avg:  2m 14s | Max:  7m 19s | Hits:  96%/1680  
  🟩 Test               Pass: 100%/8   | Total: 32m 09s | Avg:  4m 01s | Max:  4m 22s | Hits:  97%/288   
🟩 sm
  🟩 90                 Pass: 100%/1   | Total:  1m 54s | Avg:  1m 54s | Max:  1m 54s | Hits:  94%/36    
  🟩 90a                Pass: 100%/1   | Total:  2m 13s | Avg:  2m 13s | Max:  2m 13s | Hits:  94%/36    
🟩 std
  🟩 17                 Pass: 100%/31  | Total:  1h 11m | Avg:  2m 17s | Max:  4m 22s | Hits:  97%/1116  
  🟩 20                 Pass: 100%/24  | Total:  1h 06m | Avg:  2m 46s | Max:  7m 19s | Hits:  96%/852

🟩 cccl: Pass: 100%/4 | Total: 16m 30s | Avg: 4m 07s | Max: 4m 37s

🟩 cpu
  🟩 amd64              Pass: 100%/4   | Total: 16m 30s | Avg:  4m 07s | Max:  4m 37s
🟩 ctk
  🟩 11.1               Pass: 100%/2   | Total:  7m 16s | Avg:  3m 38s | Max:  3m 53s
  🟩 12.5               Pass: 100%/2   | Total:  9m 14s | Avg:  4m 37s | Max:  4m 37s
🟩 cudacxx
  🟩 nvcc11.1           Pass: 100%/2   | Total:  7m 16s | Avg:  3m 38s | Max:  3m 53s
  🟩 nvcc12.5           Pass: 100%/2   | Total:  9m 14s | Avg:  4m 37s | Max:  4m 37s
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/4   | Total: 16m 30s | Avg:  4m 07s | Max:  4m 37s
🟩 cxx
  🟩 Clang9             Pass: 100%/1   | Total:  3m 53s | Avg:  3m 53s | Max:  3m 53s
  🟩 Clang17            Pass: 100%/1   | Total:  4m 37s | Avg:  4m 37s | Max:  4m 37s
  🟩 GCC6               Pass: 100%/1   | Total:  3m 23s | Avg:  3m 23s | Max:  3m 23s
  🟩 GCC13              Pass: 100%/1   | Total:  4m 37s | Avg:  4m 37s | Max:  4m 37s
🟩 cxx_family
  🟩 Clang              Pass: 100%/2   | Total:  8m 30s | Avg:  4m 15s | Max:  4m 37s
  🟩 GCC                Pass: 100%/2   | Total:  8m 00s | Avg:  4m 00s | Max:  4m 37s
🟩 gpu
  🟩 v100               Pass: 100%/4   | Total: 16m 30s | Avg:  4m 07s | Max:  4m 37s
🟩 jobs
  🟩 Infra              Pass: 100%/4   | Total: 16m 30s | Avg:  4m 07s | Max:  4m 37s

🟩 pycuda: Pass: 100%/1 | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s

🟩 cpu
  🟩 amd64              Pass: 100%/1   | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s
🟩 ctk
  🟩 12.5               Pass: 100%/1   | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s
🟩 cudacxx
  🟩 nvcc12.5           Pass: 100%/1   | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s
🟩 cudacxx_family
  🟩 nvcc               Pass: 100%/1   | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s
🟩 cxx
  🟩 GCC13              Pass: 100%/1   | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s
🟩 cxx_family
  🟩 GCC                Pass: 100%/1   | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s
🟩 gpu
  🟩 v100               Pass: 100%/1   | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s
🟩 jobs
  🟩 Test               Pass: 100%/1   | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s

👃 Inspect Changes

Modifications in project?

	Project
+/-	CCCL Infrastructure
	libcu++
	CUB
	Thrust
	CUDA Experimental
	pycuda

Modifications in project or dependencies?

	Project
+/-	CCCL Infrastructure
+/-	libcu++
+/-	CUB
+/-	Thrust
+/-	CUDA Experimental
+/-	pycuda

🏃‍ Runner counts (total jobs: 421)

#	Runner
305	`linux-amd64-cpu16`
65	`linux-amd64-gpu-v100-latest-1`
28	`linux-arm64-cpu16`
23	`windows-amd64-cpu16`

Bench analysis: Print all top rows when asked for

b57c0f6

bernhardmgruber commented Jul 26, 2024

View reviewed changes

bernhardmgruber requested a review from gevtushenko July 26, 2024 15:47

bernhardmgruber added the benchmark Feature related to benchmarking our libraries label Jul 26, 2024

bernhardmgruber marked this pull request as ready for review July 26, 2024 15:47

bernhardmgruber requested a review from a team as a code owner July 26, 2024 15:47

bernhardmgruber requested a review from miscco July 26, 2024 15:47

gevtushenko approved these changes Jul 26, 2024

View reviewed changes

bernhardmgruber enabled auto-merge (squash) July 26, 2024 20:18

bernhardmgruber merged commit ad57b1e into NVIDIA:main Jul 27, 2024
438 of 439 checks passed

bernhardmgruber deleted the top_n branch July 29, 2024 07:41

pciolkosz pushed a commit to pciolkosz/cccl that referenced this pull request Aug 4, 2024

Bench analysis: Print all top rows when asked for (NVIDIA#2089)

18e1d3d

pciolkosz pushed a commit to pciolkosz/cccl that referenced this pull request Aug 4, 2024

Bench analysis: Print all top rows when asked for (NVIDIA#2089)

4cf3c26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Benchmark analysis: Print all top rows when asked for #2089

Benchmark analysis: Print all top rows when asked for #2089

bernhardmgruber commented Jul 26, 2024 •

edited

Loading

bernhardmgruber Jul 26, 2024

github-actions bot commented Jul 27, 2024

🟩 cub: Pass: 100%/131 | Total: 1d 11h | Avg: 16m 16s | Max: 56m 07s | Hits: 98%/111124

🟩 thrust: Pass: 100%/118 | Total: 17h 53m | Avg: 9m 05s | Max: 52m 18s | Hits: 96%/138912

🟩 libcudacxx: Pass: 100%/112 | Total: 13h 27m | Avg: 7m 12s | Max: 32m 48s | Hits: 95%/273250

🟩 cudax: Pass: 100%/55 | Total: 2h 17m | Avg: 2m 30s | Max: 7m 19s | Hits: 96%/1968

🟩 cccl: Pass: 100%/4 | Total: 16m 30s | Avg: 4m 07s | Max: 4m 37s

🟩 pycuda: Pass: 100%/1 | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s

👃 Inspect Changes

Modifications in project?

Modifications in project or dependencies?

🏃‍ Runner counts (total jobs: 421)

		with pd.option_context('display.max_rows', None):
		print(extract_scores(case_dfs).head(N))

Benchmark analysis: Print all top rows when asked for #2089

Benchmark analysis: Print all top rows when asked for #2089

Conversation

bernhardmgruber commented Jul 26, 2024 • edited Loading

bernhardmgruber Jul 26, 2024

Choose a reason for hiding this comment

github-actions bot commented Jul 27, 2024

🟩 cub: Pass: 100%/131 | Total: 1d 11h | Avg: 16m 16s | Max: 56m 07s | Hits: 98%/111124

🟩 thrust: Pass: 100%/118 | Total: 17h 53m | Avg: 9m 05s | Max: 52m 18s | Hits: 96%/138912

🟩 libcudacxx: Pass: 100%/112 | Total: 13h 27m | Avg: 7m 12s | Max: 32m 48s | Hits: 95%/273250

🟩 cudax: Pass: 100%/55 | Total: 2h 17m | Avg: 2m 30s | Max: 7m 19s | Hits: 96%/1968

🟩 cccl: Pass: 100%/4 | Total: 16m 30s | Avg: 4m 07s | Max: 4m 37s

🟩 pycuda: Pass: 100%/1 | Total: 12m 10s | Avg: 12m 10s | Max: 12m 10s

👃 Inspect Changes

Modifications in project?

Modifications in project or dependencies?

🏃‍ Runner counts (total jobs: 421)

bernhardmgruber commented Jul 26, 2024 •

edited

Loading