Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compute-sanitizer testing to CI. #1879

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

alliepiper
Copy link
Collaborator

@alliepiper alliepiper commented Jun 18, 2024

compute-sanitizer offers four sanitizer tools: memcheck, synccheck, racecheck, and initcheck. Only one tool can be enabled per execution.

CUB

Initial exploration of time cost, feasibility, and implementation will be done while adding coverage to CUB. CUB splits its tests into 4 subsets: HostLaunch, DeviceLaunch, GraphCapture, and TestGPU. Each of these will be tested with each of the four compute-sanitizer tools, giving a total of 16 possible jobs:

Sanitizer TestSet Execution Time Result Notes
memcheck TestGPU 3h 25m #1894, #1898, #1895, #1896, #1897, #1899
memcheck HostLaunch 1hr 56m #1888
memcheck DeviceLaunch 2hr 1m #1888
memcheck GraphCapture 1h 46m #1888
racecheck TestGPU 5h 5m #1902, #1903
racecheck HostLaunch 6h+ Timeout; #1891, #1892, #1893
racecheck DeviceLaunch N/A 🚫 Disabled; CDP not supported.
racecheck GraphCapture 6h+ Timeout; #1892, #1906
initcheck TestGPU 5h 3m #1889
initcheck HostLaunch 6h+ Timeout; #1889, #1890, #1891
initcheck DeviceLaunch N/A 🚫 Disabled; CDP not supported.
initcheck GraphCapture 6h+ Timeout; #1889, #1891, #1907, #1909, #1910
synccheck TestGPU 24m #1887
synccheck HostLaunch 57m
synccheck DeviceLaunch N/A 🚫 Disabled; CDP not supported.
synccheck GraphCapture 51m

Copy link
Contributor

🟨 CI finished in 6h 37m: Pass: 50%/2 | Total: 6h 37m | Avg: 3h 18m | Max: 6h 00m | Hits: 10%/856
  • 🟨 cub: Pass: 50%/2 | Total: 6h 37m | Avg: 3h 18m | Max: 6h 00m | Hits: 10%/856

    🚨 jobs: ComputeSanitizer 🚨
      🟩 Build              Pass: 100%/1   | Total: 37m 10s | Avg: 37m 10s | Max: 37m 10s | Hits:  10%/856   
      🔥 ComputeSanitizer   Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:  50%/2   | Total:  6h 37m | Avg:  3h 18m | Max:  6h 00m | Hits:  10%/856   
    🟨 cpu
      🟨 amd64              Pass:  50%/2   | Total:  6h 37m | Avg:  3h 18m | Max:  6h 00m | Hits:  10%/856   
    🟨 ctk
      🟨 12.4               Pass:  50%/2   | Total:  6h 37m | Avg:  3h 18m | Max:  6h 00m | Hits:  10%/856   
    🟨 cudacxx
      🟨 nvcc12.4           Pass:  50%/2   | Total:  6h 37m | Avg:  3h 18m | Max:  6h 00m | Hits:  10%/856   
    🟨 cudacxx_family
      🟨 nvcc               Pass:  50%/2   | Total:  6h 37m | Avg:  3h 18m | Max:  6h 00m | Hits:  10%/856   
    🟨 cxx
      🟨 GCC12              Pass:  50%/2   | Total:  6h 37m | Avg:  3h 18m | Max:  6h 00m | Hits:  10%/856   
    🟨 cxx_family
      🟨 GCC                Pass:  50%/2   | Total:  6h 37m | Avg:  3h 18m | Max:  6h 00m | Hits:  10%/856   
    🟨 gpu
      🟨 v100               Pass:  50%/2   | Total:  6h 37m | Avg:  3h 18m | Max:  6h 00m | Hits:  10%/856   
    🟨 std
      🟨 17                 Pass:  50%/2   | Total:  6h 37m | Avg:  3h 18m | Max:  6h 00m | Hits:  10%/856   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute-sanitizer"
    ],
    "project": "cub",
    "std": 17,
    "ctk": "curr",
    "cxx": [
      "gcc12"
    ],
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 2)

# Runner
1 linux-amd64-cpu16
1 linux-amd64-gpu-v100-latest-1

Copy link
Contributor

🟨 CI finished in 34m 14s: Pass: 5%/17 | Total: 2h 21m | Avg: 8m 20s | Max: 22m 40s | Hits: 31%/856
  • 🟨 cub: Pass: 5%/17 | Total: 2h 21m | Avg: 8m 20s | Max: 22m 40s | Hits: 31%/856

    🟨 jobs
      🟩 Build              Pass: 100%/1   | Total: 22m 40s | Avg: 22m 40s | Max: 22m 40s | Hits:  31%/856   
      🟥 DeviceLaunch-CS-InitCheck Pass:   0%/1   | Total:  8m 26s | Avg:  8m 26s | Max:  8m 26s
      🟥 DeviceLaunch-CS-MemCheck Pass:   0%/1   | Total:  8m 21s | Avg:  8m 21s | Max:  8m 21s
      🟥 DeviceLaunch-CS-RaceCheck Pass:   0%/1   | Total:  7m 52s | Avg:  7m 52s | Max:  7m 52s
      🟥 DeviceLaunch-CS-SyncCheck Pass:   0%/1   | Total:  3m 22s | Avg:  3m 22s | Max:  3m 22s
      🟥 GraphCapture-CS-InitCheck Pass:   0%/1   | Total:  6m 59s | Avg:  6m 59s | Max:  6m 59s
      🟥 GraphCapture-CS-MemCheck Pass:   0%/1   | Total:  8m 53s | Avg:  8m 53s | Max:  8m 53s
      🟥 GraphCapture-CS-RaceCheck Pass:   0%/1   | Total:  8m 07s | Avg:  8m 07s | Max:  8m 07s
      🟥 GraphCapture-CS-SyncCheck Pass:   0%/1   | Total:  4m 54s | Avg:  4m 54s | Max:  4m 54s
      🟥 HostLaunch-CS-InitCheck Pass:   0%/1   | Total:  9m 29s | Avg:  9m 29s | Max:  9m 29s
      🟥 HostLaunch-CS-MemCheck Pass:   0%/1   | Total:  9m 26s | Avg:  9m 26s | Max:  9m 26s
      🟥 HostLaunch-CS-RaceCheck Pass:   0%/1   | Total:  6m 53s | Avg:  6m 53s | Max:  6m 53s
      🟥 HostLaunch-CS-SyncCheck Pass:   0%/1   | Total:  6m 06s | Avg:  6m 06s | Max:  6m 06s
      🟥 TestGPU-CS-InitCheck Pass:   0%/1   | Total:  7m 54s | Avg:  7m 54s | Max:  7m 54s
      🟥 TestGPU-CS-MemCheck Pass:   0%/1   | Total:  8m 09s | Avg:  8m 09s | Max:  8m 09s
      🟥 TestGPU-CS-RaceCheck Pass:   0%/1   | Total:  9m 30s | Avg:  9m 30s | Max:  9m 30s
      🟥 TestGPU-CS-SyncCheck Pass:   0%/1   | Total:  4m 54s | Avg:  4m 54s | Max:  4m 54s
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:   5%/17  | Total:  2h 21m | Avg:  8m 20s | Max: 22m 40s | Hits:  31%/856   
    🟨 cpu
      🟨 amd64              Pass:   5%/17  | Total:  2h 21m | Avg:  8m 20s | Max: 22m 40s | Hits:  31%/856   
    🟨 ctk
      🟨 12.4               Pass:   5%/17  | Total:  2h 21m | Avg:  8m 20s | Max: 22m 40s | Hits:  31%/856   
    🟨 cudacxx
      🟨 nvcc12.4           Pass:   5%/17  | Total:  2h 21m | Avg:  8m 20s | Max: 22m 40s | Hits:  31%/856   
    🟨 cudacxx_family
      🟨 nvcc               Pass:   5%/17  | Total:  2h 21m | Avg:  8m 20s | Max: 22m 40s | Hits:  31%/856   
    🟨 cxx
      🟨 GCC13              Pass:   5%/17  | Total:  2h 21m | Avg:  8m 20s | Max: 22m 40s | Hits:  31%/856   
    🟨 cxx_family
      🟨 GCC                Pass:   5%/17  | Total:  2h 21m | Avg:  8m 20s | Max: 22m 40s | Hits:  31%/856   
    🟨 gpu
      🟨 v100               Pass:   5%/17  | Total:  2h 21m | Avg:  8m 20s | Max: 22m 40s | Hits:  31%/856   
    🟨 std
      🟨 17                 Pass:   5%/17  | Total:  2h 21m | Avg:  8m 20s | Max: 22m 40s | Hits:  31%/856   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_sanitizer"
    ],
    "project": "cub",
    "std": 17,
    "cxx": "gcc13",
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 17)

# Runner
16 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

Copy link
Contributor

🟨 CI finished in 6h 41m: Pass: 5%/17 | Total: 2d 19h | Avg: 3h 56m | Max: 6h 00m | Hits: 99%/856
  • 🟨 cub: Pass: 5%/17 | Total: 2d 19h | Avg: 3h 56m | Max: 6h 00m | Hits: 99%/856

    🟨 jobs
      🟩 Build              Pass: 100%/1   | Total:  5m 39s | Avg:  5m 39s | Max:  5m 39s | Hits:  99%/856   
      🟥 DeviceLaunch-CS-InitCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 DeviceLaunch-CS-MemCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 DeviceLaunch-CS-RaceCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 DeviceLaunch-CS-SyncCheck Pass:   0%/1   | Total:  4m 41s | Avg:  4m 41s | Max:  4m 41s
      🟥 GraphCapture-CS-InitCheck Pass:   0%/1   | Total:  3m 58s | Avg:  3m 58s | Max:  3m 58s
      🟥 GraphCapture-CS-MemCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 GraphCapture-CS-RaceCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 GraphCapture-CS-SyncCheck Pass:   0%/1   | Total:  2h 34m | Avg:  2h 34m | Max:  2h 34m
      🟥 HostLaunch-CS-InitCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 HostLaunch-CS-MemCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 HostLaunch-CS-RaceCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 HostLaunch-CS-SyncCheck Pass:   0%/1   | Total:  9m 00s | Avg:  9m 00s | Max:  9m 00s
      🟥 TestGPU-CS-InitCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 TestGPU-CS-MemCheck Pass:   0%/1   | Total:  3h 30m | Avg:  3h 30m | Max:  3h 30m
      🟥 TestGPU-CS-RaceCheck Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 TestGPU-CS-SyncCheck Pass:   0%/1   | Total: 29m 24s | Avg: 29m 24s | Max: 29m 24s
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:   5%/17  | Total:  2d 19h | Avg:  3h 56m | Max:  6h 00m | Hits:  99%/856   
    🟨 cpu
      🟨 amd64              Pass:   5%/17  | Total:  2d 19h | Avg:  3h 56m | Max:  6h 00m | Hits:  99%/856   
    🟨 ctk
      🟨 12.4               Pass:   5%/17  | Total:  2d 19h | Avg:  3h 56m | Max:  6h 00m | Hits:  99%/856   
    🟨 cudacxx
      🟨 nvcc12.4           Pass:   5%/17  | Total:  2d 19h | Avg:  3h 56m | Max:  6h 00m | Hits:  99%/856   
    🟨 cudacxx_family
      🟨 nvcc               Pass:   5%/17  | Total:  2d 19h | Avg:  3h 56m | Max:  6h 00m | Hits:  99%/856   
    🟨 cxx
      🟨 GCC13              Pass:   5%/17  | Total:  2d 19h | Avg:  3h 56m | Max:  6h 00m | Hits:  99%/856   
    🟨 cxx_family
      🟨 GCC                Pass:   5%/17  | Total:  2d 19h | Avg:  3h 56m | Max:  6h 00m | Hits:  99%/856   
    🟨 gpu
      🟨 v100               Pass:   5%/17  | Total:  2d 19h | Avg:  3h 56m | Max:  6h 00m | Hits:  99%/856   
    🟨 std
      🟨 17                 Pass:   5%/17  | Total:  2d 19h | Avg:  3h 56m | Max:  6h 00m | Hits:  99%/856   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_sanitizer"
    ],
    "project": "cub",
    "std": 17,
    "cxx": "gcc13",
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 17)

# Runner
16 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

Copy link
Contributor

🟨 CI finished in 2h 48m: Pass: 20%/5 | Total: 7h 49m | Avg: 1h 33m | Max: 2h 41m | Hits: 99%/856
  • 🟨 cub: Pass: 20%/5 | Total: 7h 49m | Avg: 1h 33m | Max: 2h 41m | Hits: 99%/856

    🟨 jobs
      🟩 Build              Pass: 100%/1   | Total:  5m 47s | Avg:  5m 47s | Max:  5m 47s | Hits:  99%/856   
      🟥 DeviceLaunch-CS-SyncCheck Pass:   0%/1   | Total:  2h 18m | Avg:  2h 18m | Max:  2h 18m
      🟥 GraphCapture-CS-SyncCheck Pass:   0%/1   | Total:  2h 41m | Avg:  2h 41m | Max:  2h 41m
      🟥 HostLaunch-CS-SyncCheck Pass:   0%/1   | Total:  2h 18m | Avg:  2h 18m | Max:  2h 18m
      🟥 TestGPU-CS-SyncCheck Pass:   0%/1   | Total: 24m 09s | Avg: 24m 09s | Max: 24m 09s
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:  20%/5   | Total:  7h 49m | Avg:  1h 33m | Max:  2h 41m | Hits:  99%/856   
    🟨 cpu
      🟨 amd64              Pass:  20%/5   | Total:  7h 49m | Avg:  1h 33m | Max:  2h 41m | Hits:  99%/856   
    🟨 ctk
      🟨 12.4               Pass:  20%/5   | Total:  7h 49m | Avg:  1h 33m | Max:  2h 41m | Hits:  99%/856   
    🟨 cudacxx
      🟨 nvcc12.4           Pass:  20%/5   | Total:  7h 49m | Avg:  1h 33m | Max:  2h 41m | Hits:  99%/856   
    🟨 cudacxx_family
      🟨 nvcc               Pass:  20%/5   | Total:  7h 49m | Avg:  1h 33m | Max:  2h 41m | Hits:  99%/856   
    🟨 cxx
      🟨 GCC13              Pass:  20%/5   | Total:  7h 49m | Avg:  1h 33m | Max:  2h 41m | Hits:  99%/856   
    🟨 cxx_family
      🟨 GCC                Pass:  20%/5   | Total:  7h 49m | Avg:  1h 33m | Max:  2h 41m | Hits:  99%/856   
    🟨 gpu
      🟨 v100               Pass:  20%/5   | Total:  7h 49m | Avg:  1h 33m | Max:  2h 41m | Hits:  99%/856   
    🟨 std
      🟨 17                 Pass:  20%/5   | Total:  7h 49m | Avg:  1h 33m | Max:  2h 41m | Hits:  99%/856   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_sanitizer"
    ],
    "project": "cub",
    "std": 17,
    "cxx": "gcc13",
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 5)

# Runner
4 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

Copy link
Contributor

🟨 CI finished in 1h 19m: Pass: 25%/4 | Total: 2h 41m | Avg: 40m 15s | Max: 1h 13m | Hits: 99%/856
  • 🟨 cub: Pass: 25%/4 | Total: 2h 41m | Avg: 40m 15s | Max: 1h 13m | Hits: 99%/856

    🟨 jobs
      🟩 Build              Pass: 100%/1   | Total:  6m 10s | Avg:  6m 10s | Max:  6m 10s | Hits:  99%/856   
      🟥 GraphCapture-CS-SyncCheck Pass:   0%/1   | Total:  1h 13m | Avg:  1h 13m | Max:  1h 13m
      🟥 HostLaunch-CS-SyncCheck Pass:   0%/1   | Total: 56m 51s | Avg: 56m 51s | Max: 56m 51s
      🟥 TestGPU-CS-SyncCheck Pass:   0%/1   | Total: 24m 57s | Avg: 24m 57s | Max: 24m 57s
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:  25%/4   | Total:  2h 41m | Avg: 40m 15s | Max:  1h 13m | Hits:  99%/856   
    🟨 cpu
      🟨 amd64              Pass:  25%/4   | Total:  2h 41m | Avg: 40m 15s | Max:  1h 13m | Hits:  99%/856   
    🟨 ctk
      🟨 12.4               Pass:  25%/4   | Total:  2h 41m | Avg: 40m 15s | Max:  1h 13m | Hits:  99%/856   
    🟨 cudacxx
      🟨 nvcc12.4           Pass:  25%/4   | Total:  2h 41m | Avg: 40m 15s | Max:  1h 13m | Hits:  99%/856   
    🟨 cudacxx_family
      🟨 nvcc               Pass:  25%/4   | Total:  2h 41m | Avg: 40m 15s | Max:  1h 13m | Hits:  99%/856   
    🟨 cxx
      🟨 GCC13              Pass:  25%/4   | Total:  2h 41m | Avg: 40m 15s | Max:  1h 13m | Hits:  99%/856   
    🟨 cxx_family
      🟨 GCC                Pass:  25%/4   | Total:  2h 41m | Avg: 40m 15s | Max:  1h 13m | Hits:  99%/856   
    🟨 gpu
      🟨 v100               Pass:  25%/4   | Total:  2h 41m | Avg: 40m 15s | Max:  1h 13m | Hits:  99%/856   
    🟨 std
      🟨 17                 Pass:  25%/4   | Total:  2h 41m | Avg: 40m 15s | Max:  1h 13m | Hits:  99%/856   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_sanitizer"
    ],
    "project": "cub",
    "std": 17,
    "cxx": "gcc13",
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 4)

# Runner
3 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

@alliepiper alliepiper force-pushed the compute_sanitizer branch 3 times, most recently from 53fad3c to be501fb Compare June 21, 2024 00:44
Copy link
Contributor

🟨 CI finished in 6h 52m: Pass: 10%/10 | Total: 1d 20h | Avg: 4h 26m | Max: 6h 00m | Hits: 98%/856
  • 🟨 cub: Pass: 10%/10 | Total: 1d 20h | Avg: 4h 26m | Max: 6h 00m | Hits: 98%/856

    🟨 jobs
      🟩 Build              Pass: 100%/1   | Total:  6m 12s | Avg:  6m 12s | Max:  6m 12s | Hits:  98%/856   
      🟥 CSInit-DeviceLaunch Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSInit-GraphCapture Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSInit-HostLaunch  Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSInit-TestGPU     Pass:   0%/1   | Total:  5h 03m | Avg:  5h 03m | Max:  5h 03m
      🟥 CSMem-TestGPU      Pass:   0%/1   | Total:  3h 25m | Avg:  3h 25m | Max:  3h 25m
      🟥 CSRace-DeviceLaunch Pass:   0%/1   | Total: 48m 16s | Avg: 48m 16s | Max: 48m 16s
      🟥 CSRace-GraphCapture Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSRace-HostLaunch  Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSRace-TestGPU     Pass:   0%/1   | Total:  5h 05m | Avg:  5h 05m | Max:  5h 05m
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:  10%/10  | Total:  1d 20h | Avg:  4h 26m | Max:  6h 00m | Hits:  98%/856   
    🟨 cpu
      🟨 amd64              Pass:  10%/10  | Total:  1d 20h | Avg:  4h 26m | Max:  6h 00m | Hits:  98%/856   
    🟨 ctk
      🟨 12.4               Pass:  10%/10  | Total:  1d 20h | Avg:  4h 26m | Max:  6h 00m | Hits:  98%/856   
    🟨 cudacxx
      🟨 nvcc12.4           Pass:  10%/10  | Total:  1d 20h | Avg:  4h 26m | Max:  6h 00m | Hits:  98%/856   
    🟨 cudacxx_family
      🟨 nvcc               Pass:  10%/10  | Total:  1d 20h | Avg:  4h 26m | Max:  6h 00m | Hits:  98%/856   
    🟨 cxx
      🟨 GCC13              Pass:  10%/10  | Total:  1d 20h | Avg:  4h 26m | Max:  6h 00m | Hits:  98%/856   
    🟨 cxx_family
      🟨 GCC                Pass:  10%/10  | Total:  1d 20h | Avg:  4h 26m | Max:  6h 00m | Hits:  98%/856   
    🟨 gpu
      🟨 v100               Pass:  10%/10  | Total:  1d 20h | Avg:  4h 26m | Max:  6h 00m | Hits:  98%/856   
    🟨 std
      🟨 17                 Pass:  10%/10  | Total:  1d 20h | Avg:  4h 26m | Max:  6h 00m | Hits:  98%/856   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_mem_nolid",
      "compute_race_nolid",
      "compute_race_lid0",
      "compute_race_lid1",
      "compute_race_lid2",
      "compute_init_nolid",
      "compute_init_lid0",
      "compute_init_lid1",
      "compute_init_lid2"
    ],
    "project": "cub",
    "std": 17,
    "cxx": "gcc13",
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 10)

# Runner
9 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

Copy link
Contributor

🟨 CI finished in 6h 37m: Pass: 12%/8 | Total: 1d 14h | Avg: 4h 52m | Max: 6h 00m | Hits: 9%/852
  • 🟨 cub: Pass: 12%/8 | Total: 1d 14h | Avg: 4h 52m | Max: 6h 00m | Hits: 9%/852

    🟨 jobs
      🟩 Build              Pass: 100%/1   | Total: 36m 38s | Avg: 36m 38s | Max: 36m 38s | Hits:   9%/852   
      🟥 CSInit-GraphCapture Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSInit-HostLaunch  Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSInit-TestGPU     Pass:   0%/1   | Total:  5h 05m | Avg:  5h 05m | Max:  5h 05m
      🟥 CSMem-TestGPU      Pass:   0%/1   | Total:  4h 23m | Avg:  4h 23m | Max:  4h 23m
      🟥 CSRace-GraphCapture Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSRace-HostLaunch  Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSRace-TestGPU     Pass:   0%/1   | Total:  4h 52m | Avg:  4h 52m | Max:  4h 52m
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:  12%/8   | Total:  1d 14h | Avg:  4h 52m | Max:  6h 00m | Hits:   9%/852   
    🟨 cpu
      🟨 amd64              Pass:  12%/8   | Total:  1d 14h | Avg:  4h 52m | Max:  6h 00m | Hits:   9%/852   
    🟨 ctk
      🟨 12.4               Pass:  12%/8   | Total:  1d 14h | Avg:  4h 52m | Max:  6h 00m | Hits:   9%/852   
    🟨 cudacxx
      🟨 nvcc12.4           Pass:  12%/8   | Total:  1d 14h | Avg:  4h 52m | Max:  6h 00m | Hits:   9%/852   
    🟨 cudacxx_family
      🟨 nvcc               Pass:  12%/8   | Total:  1d 14h | Avg:  4h 52m | Max:  6h 00m | Hits:   9%/852   
    🟨 cxx
      🟨 GCC13              Pass:  12%/8   | Total:  1d 14h | Avg:  4h 52m | Max:  6h 00m | Hits:   9%/852   
    🟨 cxx_family
      🟨 GCC                Pass:  12%/8   | Total:  1d 14h | Avg:  4h 52m | Max:  6h 00m | Hits:   9%/852   
    🟨 gpu
      🟨 v100               Pass:  12%/8   | Total:  1d 14h | Avg:  4h 52m | Max:  6h 00m | Hits:   9%/852   
    🟨 std
      🟨 17                 Pass:  12%/8   | Total:  1d 14h | Avg:  4h 52m | Max:  6h 00m | Hits:   9%/852   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_mem_nolid",
      "compute_race_nolid",
      "compute_race_lid0",
      "compute_race_lid2",
      "compute_init_nolid",
      "compute_init_lid0",
      "compute_init_lid2"
    ],
    "project": "cub",
    "std": 17,
    "cxx": "gcc13",
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 8)

# Runner
7 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

cub/test/run_test.cmake Outdated Show resolved Hide resolved
@alliepiper alliepiper force-pushed the compute_sanitizer branch 2 times, most recently from 583affb to 20347c7 Compare June 28, 2024 15:46
Copy link
Contributor

🟨 CI finished in 2h 31m: Pass: 83%/6 | Total: 53m 29s | Avg: 8m 54s | Max: 14m 08s | Hits: 99%/4260
  • 🟨 cub: Pass: 83%/6 | Total: 53m 29s | Avg: 8m 54s | Max: 14m 08s | Hits: 99%/4260

    🚨 jobs: CSMem-TestGPU 🚨
      🟩 Build              Pass: 100%/1   | Total:  5m 17s | Avg:  5m 17s | Max:  5m 17s | Hits:  99%/852   
      🟩 CSInit-GraphCapture Pass: 100%/1   | Total:  6m 59s | Avg:  6m 59s | Max:  6m 59s | Hits:  99%/852   
      🟩 CSInit-HostLaunch  Pass: 100%/1   | Total:  8m 24s | Avg:  8m 24s | Max:  8m 24s | Hits:  99%/852   
      🔥 CSMem-TestGPU      Pass:   0%/1   | Total: 14m 08s | Avg: 14m 08s | Max: 14m 08s
      🟩 CSRace-GraphCapture Pass: 100%/1   | Total:  9m 23s | Avg:  9m 23s | Max:  9m 23s | Hits:  99%/852   
      🟩 CSRace-HostLaunch  Pass: 100%/1   | Total:  9m 18s | Avg:  9m 18s | Max:  9m 18s | Hits:  99%/852   
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:  83%/6   | Total: 53m 29s | Avg:  8m 54s | Max: 14m 08s | Hits:  99%/4260  
    🟨 cpu
      🟨 amd64              Pass:  83%/6   | Total: 53m 29s | Avg:  8m 54s | Max: 14m 08s | Hits:  99%/4260  
    🟨 ctk
      🟨 12.4               Pass:  83%/6   | Total: 53m 29s | Avg:  8m 54s | Max: 14m 08s | Hits:  99%/4260  
    🟨 cudacxx
      🟨 nvcc12.4           Pass:  83%/6   | Total: 53m 29s | Avg:  8m 54s | Max: 14m 08s | Hits:  99%/4260  
    🟨 cudacxx_family
      🟨 nvcc               Pass:  83%/6   | Total: 53m 29s | Avg:  8m 54s | Max: 14m 08s | Hits:  99%/4260  
    🟨 cxx
      🟨 GCC13              Pass:  83%/6   | Total: 53m 29s | Avg:  8m 54s | Max: 14m 08s | Hits:  99%/4260  
    🟨 cxx_family
      🟨 GCC                Pass:  83%/6   | Total: 53m 29s | Avg:  8m 54s | Max: 14m 08s | Hits:  99%/4260  
    🟨 gpu
      🟨 v100               Pass:  83%/6   | Total: 53m 29s | Avg:  8m 54s | Max: 14m 08s | Hits:  99%/4260  
    🟨 std
      🟨 17                 Pass:  83%/6   | Total: 53m 29s | Avg:  8m 54s | Max: 14m 08s | Hits:  99%/4260  
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_mem_nolid",
      "compute_race_lid0",
      "compute_race_lid2",
      "compute_init_lid0",
      "compute_init_lid2"
    ],
    "project": "cub",
    "std": 17,
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 6)

# Runner
5 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

Copy link
Contributor

🟨 CI finished in 6h 07m: Pass: 20%/5 | Total: 1d 00h | Avg: 4h 49m | Max: 6h 00m | Hits: 98%/852
  • 🟨 cub: Pass: 20%/5 | Total: 1d 00h | Avg: 4h 49m | Max: 6h 00m | Hits: 98%/852

    🟨 jobs
      🟩 Build              Pass: 100%/1   | Total:  6m 30s | Avg:  6m 30s | Max:  6m 30s | Hits:  98%/852   
      🟥 CSInit-GraphCapture Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSInit-HostLaunch  Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSRace-GraphCapture Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
      🟥 CSRace-HostLaunch  Pass:   0%/1   | Total:  6h 00m | Avg:  6h 00m | Max:  6h 00m
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:  20%/5   | Total:  1d 00h | Avg:  4h 49m | Max:  6h 00m | Hits:  98%/852   
    🟨 cpu
      🟨 amd64              Pass:  20%/5   | Total:  1d 00h | Avg:  4h 49m | Max:  6h 00m | Hits:  98%/852   
    🟨 ctk
      🟨 12.4               Pass:  20%/5   | Total:  1d 00h | Avg:  4h 49m | Max:  6h 00m | Hits:  98%/852   
    🟨 cudacxx
      🟨 nvcc12.4           Pass:  20%/5   | Total:  1d 00h | Avg:  4h 49m | Max:  6h 00m | Hits:  98%/852   
    🟨 cudacxx_family
      🟨 nvcc               Pass:  20%/5   | Total:  1d 00h | Avg:  4h 49m | Max:  6h 00m | Hits:  98%/852   
    🟨 cxx
      🟨 GCC13              Pass:  20%/5   | Total:  1d 00h | Avg:  4h 49m | Max:  6h 00m | Hits:  98%/852   
    🟨 cxx_family
      🟨 GCC                Pass:  20%/5   | Total:  1d 00h | Avg:  4h 49m | Max:  6h 00m | Hits:  98%/852   
    🟨 gpu
      🟨 v100               Pass:  20%/5   | Total:  1d 00h | Avg:  4h 49m | Max:  6h 00m | Hits:  98%/852   
    🟨 std
      🟨 17                 Pass:  20%/5   | Total:  1d 00h | Avg:  4h 49m | Max:  6h 00m | Hits:  98%/852   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_race_lid0",
      "compute_race_lid2",
      "compute_init_lid0",
      "compute_init_lid2"
    ],
    "project": "cub",
    "std": 17,
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 5)

# Runner
4 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

[skip-vdc][skip-docs][skip-rapids]
[skip-vdc][skip-docs][skip-rapids]
Copy link
Contributor

🟨 CI finished in 4h 58m: Pass: 20%/5 | Total: 15h 35m | Avg: 3h 07m | Max: 4h 20m | Hits: 6%/853
  • 🟨 cub: Pass: 20%/5 | Total: 15h 35m | Avg: 3h 07m | Max: 4h 20m | Hits: 6%/853

    🟨 jobs
      🟩 Build              Pass: 100%/1   | Total: 37m 27s | Avg: 37m 27s | Max: 37m 27s | Hits:   6%/853   
      🟥 CSInit-GraphCapture Pass:   0%/1   | Total:  2h 52m | Avg:  2h 52m | Max:  2h 52m
      🟥 CSInit-HostLaunch  Pass:   0%/1   | Total:  4h 16m | Avg:  4h 16m | Max:  4h 16m
      🟥 CSRace-GraphCapture Pass:   0%/1   | Total:  3h 29m | Avg:  3h 29m | Max:  3h 29m
      🟥 CSRace-HostLaunch  Pass:   0%/1   | Total:  4h 20m | Avg:  4h 20m | Max:  4h 20m
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:  20%/5   | Total: 15h 35m | Avg:  3h 07m | Max:  4h 20m | Hits:   6%/853   
    🟨 cpu
      🟨 amd64              Pass:  20%/5   | Total: 15h 35m | Avg:  3h 07m | Max:  4h 20m | Hits:   6%/853   
    🟨 ctk
      🟨 12.5               Pass:  20%/5   | Total: 15h 35m | Avg:  3h 07m | Max:  4h 20m | Hits:   6%/853   
    🟨 cudacxx
      🟨 nvcc12.5           Pass:  20%/5   | Total: 15h 35m | Avg:  3h 07m | Max:  4h 20m | Hits:   6%/853   
    🟨 cudacxx_family
      🟨 nvcc               Pass:  20%/5   | Total: 15h 35m | Avg:  3h 07m | Max:  4h 20m | Hits:   6%/853   
    🟨 cxx
      🟨 GCC13              Pass:  20%/5   | Total: 15h 35m | Avg:  3h 07m | Max:  4h 20m | Hits:   6%/853   
    🟨 cxx_family
      🟨 GCC                Pass:  20%/5   | Total: 15h 35m | Avg:  3h 07m | Max:  4h 20m | Hits:   6%/853   
    🟨 gpu
      🟨 v100               Pass:  20%/5   | Total: 15h 35m | Avg:  3h 07m | Max:  4h 20m | Hits:   6%/853   
    🟨 std
      🟨 17                 Pass:  20%/5   | Total: 15h 35m | Avg:  3h 07m | Max:  4h 20m | Hits:   6%/853   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_race_lid0",
      "compute_race_lid2",
      "compute_init_lid0",
      "compute_init_lid2"
    ],
    "project": "cub",
    "std": 17,
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 5)

# Runner
4 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

Copy link
Contributor

🟨 CI finished in 4h 21m: Pass: 20%/5 | Total: 11h 35m | Avg: 2h 19m | Max: 3h 43m | Hits: 6%/853
  • 🟨 cub: Pass: 20%/5 | Total: 11h 35m | Avg: 2h 19m | Max: 3h 43m | Hits: 6%/853

    🟨 jobs
      🟩 Build              Pass: 100%/1   | Total: 36m 51s | Avg: 36m 51s | Max: 36m 51s | Hits:   6%/853   
      🟥 CSInit-GraphCapture Pass:   0%/1   | Total:  2h 16m | Avg:  2h 16m | Max:  2h 16m
      🟥 CSInit-HostLaunch  Pass:   0%/1   | Total:  3h 43m | Avg:  3h 43m | Max:  3h 43m
      🟥 CSRace-GraphCapture Pass:   0%/1   | Total:  1h 52m | Avg:  1h 52m | Max:  1h 52m
      🟥 CSRace-HostLaunch  Pass:   0%/1   | Total:  3h 05m | Avg:  3h 05m | Max:  3h 05m
    🟨 cmake_options
      🟨 -DCMAKE_CUDA_FLAGS=-lineinfo Pass:  20%/5   | Total: 11h 35m | Avg:  2h 19m | Max:  3h 43m | Hits:   6%/853   
    🟨 cpu
      🟨 amd64              Pass:  20%/5   | Total: 11h 35m | Avg:  2h 19m | Max:  3h 43m | Hits:   6%/853   
    🟨 ctk
      🟨 12.5               Pass:  20%/5   | Total: 11h 35m | Avg:  2h 19m | Max:  3h 43m | Hits:   6%/853   
    🟨 cudacxx
      🟨 nvcc12.5           Pass:  20%/5   | Total: 11h 35m | Avg:  2h 19m | Max:  3h 43m | Hits:   6%/853   
    🟨 cudacxx_family
      🟨 nvcc               Pass:  20%/5   | Total: 11h 35m | Avg:  2h 19m | Max:  3h 43m | Hits:   6%/853   
    🟨 cxx
      🟨 GCC13              Pass:  20%/5   | Total: 11h 35m | Avg:  2h 19m | Max:  3h 43m | Hits:   6%/853   
    🟨 cxx_family
      🟨 GCC                Pass:  20%/5   | Total: 11h 35m | Avg:  2h 19m | Max:  3h 43m | Hits:   6%/853   
    🟨 gpu
      🟨 v100               Pass:  20%/5   | Total: 11h 35m | Avg:  2h 19m | Max:  3h 43m | Hits:   6%/853   
    🟨 std
      🟨 17                 Pass:  20%/5   | Total: 11h 35m | Avg:  2h 19m | Max:  3h 43m | Hits:   6%/853   
    

🛠️ Override Matrix

[
  {
    "jobs": [
      "compute_race_lid0",
      "compute_race_lid2",
      "compute_init_lid0",
      "compute_init_lid2"
    ],
    "project": "cub",
    "std": 17,
    "cmake_options": "-DCMAKE_CUDA_FLAGS=-lineinfo"
  }
]

👃 Inspect Changes

Modifications in project?

Project
+/- CCCL Infrastructure
libcu++
+/- CUB
Thrust
CUDA Experimental

Modifications in project or dependencies?

Project
+/- CCCL Infrastructure
+/- libcu++
+/- CUB
+/- Thrust
+/- CUDA Experimental

🏃‍ Runner counts (total jobs: 5)

# Runner
4 linux-amd64-gpu-v100-latest-1
1 linux-amd64-cpu16

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

[FEA]: Run tests through compute-sanitizer in CI
1 participant