V2 Performance Signal Detected by TorchBench CI on '2.6.0.dev20240913+cu124'

[xuzhao9]

TorchBench CI has detected a performance signal.

Base PyTorch version: 2.4.0.dev20240606+cu121

Base PyTorch commit: ffaea656b5d8ff6518669494cc8f664b94f8e8b1

Affected PyTorch version: 2.6.0.dev20240913+cu124

Affected PyTorch commit: 0cdc6a8dcd7e294b01d8914385bbe45e79c1770d

Affected Tests:

• test_train[Super_SloMo-cuda-eager]: +14.63326%
• test_eval[Super_SloMo-cuda-eager]: +11.07465%
• test_train[Super_SloMo-cuda-jit]: +14.75198%
• test_eval[Super_SloMo-cuda-jit]: +11.21001%
• test_eval[alexnet-cuda-eager]: +10.51204%
• test_eval[alexnet-cuda-jit]: +13.69706%
• test_train[dcgan-cuda-eager]: +63.52450%
• test_train[dcgan-cuda-jit]: +63.64281%
• test_eval[pytorch_stargan-cpu-eager]: -13.98700%
• test_eval[pytorch_stargan-cpu-jit]: -16.99222%
• test_eval[shufflenet_v2_x1_0-cpu-eager]: +9.95003%
• test_eval[shufflenet_v2_x1_0-cuda-jit]: -27.56645%
• test_train[timm_nfnet-cpu-jit]: -10.66819%
• test_eval[timm_regnet-cuda-eager]: +170.17055%
• test_eval[timm_regnet-cuda-jit]: +355.06584%
• test_train[yolov3-cuda-eager]: -9.78973%

cc @xuzhao9

Result json:

{
  "start": "ffaea656b5d8ff6518669494cc8f664b94f8e8b1",
  "end": "0cdc6a8dcd7e294b01d8914385bbe45e79c1770d",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "f347174d61f",
      "commit1_time": "2024-08-15 23:39:07 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.25618776082992556
      },
      "commit2": "413416cf335",
      "commit2_time": "2024-08-15 23:41:39 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2959236922673881
      }
    },
    {
      "commit1": "0d2be06d94f",
      "commit1_time": "2024-08-15 23:55:09 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2984663169831038
      },
      "commit2": "c8ad5e37e84",
      "commit2_time": "2024-08-16 00:22:43 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2707436577416956
      }
    },
    {
      "commit1": "c8ad5e37e84",
      "commit1_time": "2024-08-16 00:22:43 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2707436577416956
      },
      "commit2": "d9576c94404",
      "commit2_time": "2024-08-16 00:22:43 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.24796438813209534
      }
    },
    {
      "commit1": "2eef749b313",
      "commit1_time": "2024-08-23 16:11:09 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-eager]": 0.2740244845859706
      },
      "commit2": "6eae5695466",
      "commit2_time": "2024-08-23 16:28:57 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-eager]": 0.22645769575610758
      }
    },
    {
      "commit1": "311af3b9889",
      "commit1_time": "2024-08-23 04:54:26 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-eager]": 0.23926895279437305
      },
      "commit2": "49b9f2d8b0a",
      "commit2_time": "2024-08-23 05:40:05 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-eager]": 0.2684786395169795
      }
    }
  ]
}

Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/10853296374