Higher than 1-D arrays in dot

[jakirkham]

Trying to use dot from pygpu with OpenCL. However it seems to throw an error due to the lack of collective operations. Am able to work around this using BLAS functions (e.g. gemm). After looking around in the code was able to find the error and the C function involved, but not exactly how we got to it.

Have a few questions. How are collective operations involved here (higher dimensions?)? Would it make sense to special case some ranks of arrays to avoid collective operations and go straight to BLAS operations? If I'm misunderstanding things here, please feel free to correct me.

In [1]: import pygpu

In [2]: import pygpu.blas

In [3]: import numpy

In [4]: import numpy as np

In [5]: c = pygpu.init("opencl0:1")

In [6]: a = pygpu.gpuarray.zeros((10, 11), dtype=np.float32, context=c, cls=pygpu.ndgpuarray)

In [7]: pygpu.blas.gemm(1, a, a.T, 0)
OpenCL Build Warning : Compiler build log:
<program source>:101:54: warning: double precision constant requires cl_khr_fp64, casting to single precision
      lA[ 0*localAStride ] = ( globalARow(0) >= M) ? 0.0 : A[ GET_GLOBAL_INDEX_A( globalARow(0), globalACol(0) ) ];
                                                     ^
<program source>:104:53: warning: double precision constant requires cl_khr_fp64, casting to single precision
      lB[ 0*localBStride ] = (globalBCol(0) >= N) ? 0.0 : B[ GET_GLOBAL_INDEX_B( globalBRow(0), globalBCol(0) ) ];
                                                    ^


Out[7]: 
gpuarray.array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]], dtype=float32)

In [8]: pygpu.blas.dot(a, a.T)
---------------------------------------------------------------------------
GpuArrayException                         Traceback (most recent call last)
<ipython-input-8-ee1cc1b223fb> in <module>()
----> 1 pygpu.blas.dot(a, a.T)

pygpu/blas.pyx in pygpu.blas.dot (pygpu/blas.c:2400)()

pygpu/blas.pyx in pygpu.blas.pygpu_blas_rdot (pygpu/blas.c:1713)()

GpuArrayException: ('Collectives operations not supported on OpenCL', 2)

[abergeron]

The message is most certainly an mistake somewhere. Possibly a reuse of the message from another failure (which is an issue still present on 0.6.5 that should be fixed with 0.6.6).

Can you try with the current master and see if you get the same problem?

[jakirkham]

Sure, retried with 0.6.8, which I built locally. Am getting an error related to dimensionality instead. Looks like 2-D arrays are not supported by dot. Is this expected? If so, what are your thoughts on expanding dot to N-D arrays of rank greater than 1?

In [1]: import pygpu

In [2]: import pygpu.blas

In [3]: import numpy

In [4]: import numpy as np

In [5]: c = pygpu.init("opencl0:1")

In [6]: a = pygpu.gpuarray.zeros((10, 11), dtype=np.float32, context=c, cls=pygpu.ndgpuarray)

In [7]: pygpu.blas.gemm(1, a, a.T, 0)
Out[7]: 
gpuarray.array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]], dtype=float32)

In [8]: pygpu.blas.dot(a, a.T)
---------------------------------------------------------------------------
GpuArrayException                         Traceback (most recent call last)
<ipython-input-8-ee1cc1b223fb> in <module>()
----> 1 pygpu.blas.dot(a, a.T)

pygpu/blas.pyx in pygpu.blas.dot (pygpu/blas.c:2568)()

pygpu/blas.pyx in pygpu.blas.pygpu_blas_rdot (pygpu/blas.c:1731)()

GpuArrayException: ('Wrong number of dimensions: X->nd = 2 (expected 1), Y->nd = 2 (expected 1), Z->nd = 0 (expected 0)', 2)

[nouiz]

This is the blas dot. not the numpy dot. the blas dot only handle 1d. I think we should do a pygpu.dot() to be like numpy.dot(). Does one already exist? @abergeron, would you put it elsewhere?

…

On Tue, Jun 27, 2017 at 4:40 PM jakirkham ***@***.***> wrote: Sure, retried with 0.6.8, which I built locally. Am getting an error related to dimensionality instead. Looks like 2-D arrays are not supported by dot. Is this expected? If so, what are your thoughts on expanding dot to N-D arrays of rank greater than 1? In [1]: import pygpu In [2]: import pygpu.blas In [3]: import numpy In [4]: import numpy as np In [5]: c = pygpu.init("opencl0:1") In [6]: a = pygpu.gpuarray.zeros((10, 11), dtype=np.float32, context=c, cls=pygpu.ndgpuarray) In [7]: pygpu.blas.gemm(1, a, a.T, 0) Out[7]: gpuarray.array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]], dtype=float32) In [8]: pygpu.blas.dot(a, a.T)--------------------------------------------------------------------------- GpuArrayException Traceback (most recent call last)<ipython-input-8-ee1cc1b223fb> in <module>()----> 1 pygpu.blas.dot(a, a.T) pygpu/blas.pyx in pygpu.blas.dot (pygpu/blas.c:2568)() pygpu/blas.pyx in pygpu.blas.pygpu_blas_rdot (pygpu/blas.c:1731)() GpuArrayException: ('Wrong number of dimensions: X->nd = 2 (expected 1), Y->nd = 2 (expected 1), Z->nd = 0 (expected 0)', 2) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#468 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AALC-2DQEcNtTisBtQu85O3dsgktUd7Mks5sIWi1gaJpZM4OGAcZ> .

[abergeron]

There isn't a pygpu.dot() akin to numpy yet. I welcome patches.

Otherwise, you're right this it the blas dot, not the numpy dot.