Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Cast tests on Databricks 14.3 #11530

Closed
razajafri opened this issue Sep 27, 2024 · 3 comments
Closed

Fix Cast tests on Databricks 14.3 #11530

razajafri opened this issue Sep 27, 2024 · 3 comments
Assignees
@mythrocks

Comments

@razajafri
Copy link
Collaborator

Build the plugin against the Databricks 14.3 cluster using #11467. Once built successfully run the cast tests by TESTS=cast_test.py jenkins/databricks/test.sh

The following tests fail

[gw2] [ 20%] FAILED ../../src/main/python/cast_test.py::test_cast_timestamp_to_numeric_non_ansi
@razajafri razajafri changed the title Fix Cast tests Fix Cast tests on Databricks 14.3 Sep 27, 2024
@mythrocks mythrocks self-assigned this Oct 7, 2024
@mythrocks
Copy link
Collaborator

I'm taking a look at this one. I've seen this failure in the context of Spark 4.

@mythrocks mythrocks mentioned this issue Oct 7, 2024
Merged
@mythrocks
Copy link
Collaborator

#11559 should now address this too.

mythrocks added a commit that referenced this issue Oct 9, 2024
@mythrocks
[Spark 4.0] Address test failures in cast_test.py [databricks] (#11559)
e8b78c0 
* Spark 4:  Addressed cast_test.py failures.

Fixes #11009 and #11530.

This commit addresses the test failures in cast_test.py, on Spark 4.0.
These generally have to do with changes in behaviour of Spark when
ANSI mode is enabled.  In these cases, the tests have been split out into ANSI=on and ANSI=off.  

The bugs uncovered from the tests have been spun into their own issues;  fixing all of them was beyond the scope of this change.

Signed-off-by: MithunR <[email protected]>
@mythrocks
Copy link
Collaborator

This is addressed by #11559.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mythrocks mythrocks

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mythrocks @razajafri