Semantically, lda should be K for a row major matrix with shape (MxK), similarly, ldc should be M for a column major matrix with shape (MxN)

[OuyeXie]

Corrected leading dimensions for A and C. Details as follows:

• K: Leading dimension of A
  • For row-major A[M, K], leading dim = K
  • cuBLAS sees this as column-major with stride K (with CUBLAS_OP_T)
• K: Leading dimension of B
  • For column-major B[K, N], leading dim = K
• M: Leading dimension of C
  • For column-major C[M, N], leading dim = M

Tested with M=256, N=128, K=64, locally.

[tspeterkim]

Also found this bug. Wonder how it was working before? From my testing, seems to be fine for square matrices but for rectangle shapes, cublas complains at runtime:

On entry to GEMM_EX  parameter number 9 had an illegal value
CUBLAS error: 7