-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Copy the spark-specific implementation of murmur32 hash from cudf into spark-rapids-jni #1246
Conversation
…mur hash instead of the cudf version. Brought over cpp and java tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Marking this as a draft since it depends upon #1244.
This needs a signoff commit per the contributor guidelines.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, code is largely identical to the cudf code with minimal refactoring. Only concern is if the to_java_bigdecimal
belongs with the decimal utils, but I think that's a nit.
build |
1 similar comment
build |
merge |
This duplicates the implementation, cpp tests and java tests for the spark-specific murmur32 hash done in cudf by @rwlee. The jni bindings now point to this implementation instead of cudf so in theory we could deprecate what's left in cudf.
One thing I didn't do was trim out the "Spark" name prefixes scattered around the code. Since the code is properly in Spark-land now there's no real need to be so verbose. If people want, I can clean it up.
Includes a small refactoring of the java decimal128 conversion code (see
to_java_bigdecimal
inhash.cuh
) which will also be used by xxhash64.Dependent on #1244