GitHub Community
Cumulative summation wrong/overflow results #21468
Answered
by
hendabdelaziz
hendabdelaziz
asked this question in
General
-
|
I am working on numba cuda, i am trying to calculate the cumulation of a matrix and whenever the value exceed 127 i got overflow / wrong values the whole algorithm done within shared memory. However i tried to predefine shared memory as uibt8, 16,32 and float32 |
Beta Was this translation helpful? Give feedback.
Answered by
hendabdelaziz
Nov 19, 2020
Replies: 1 comment
-
|
It is both the cpu and gpu sides. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It is both the cpu and gpu sides.
I just only checked the cuda side only the calculations but forgot to change the cpu sideso the output appeared only values limited to its predefined.