[GLUTEN-10134][VL] (STORE_ASSIGNMENT_POLICY defaults to ANSI) Preserve store assignment cast modes#12051
Conversation
|
Run Gluten Clickhouse CI on x86 |
|
Could you please review this? @rui-mo |
rui-mo
left a comment
rui-mo
left a comment
There was a problem hiding this comment.
Do we have any background to support this policy, and would it make sense to add some tests covering the configurable behaviors under different policy settings?
| type == BIGINT(); | ||
| } | ||
|
|
||
| bool isAnsiSupported(const TypePtr& fromType, const TypePtr& toType) { |
There was a problem hiding this comment.
There's also a check on the Velox side. Will we maintain the ANSI support check only here?
There was a problem hiding this comment.
This is intended to mirror the existing Velox ANSI support side check not to replace it, since that check is private today, I added this local helper for the expression level ANSI/legacy cast mode. I’ll add a comment to make sure we keep it aligned with Velox
| } | ||
| } | ||
|
|
||
| testGluten("storeAssignmentPolicy default ANSI is independent from ANSI mode") { |
There was a problem hiding this comment.
This doesn't appear to be a variant of a Spark test. Can we move it to a new suite under backends-velox/?
|
|
||
| spark.sql("CREATE TABLE store_assignment_ansi (c INT) USING PARQUET") | ||
| intercept[Exception] { | ||
| spark.sql("INSERT INTO store_assignment_ansi SELECT '2147483648'").collect() |
There was a problem hiding this comment.
Can we add a check for the exception message to confirm the expected exception is thrown?
My understanding is that Spark applies |
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
3 participants
What changes are proposed in this pull request?
This PR preserves Spark cast eval mode when Gluten translates casts to Velox, so store assignment casts keep ANSI behavior even when
spark.sql.ansi.enabled=false.#10134
TRY_CASTas Velox try-cast.STORE_ASSIGNMENT_POLICY=ANSI.How was this patch tested?
Was this patch authored or co-authored using generative AI tooling?