Skip to content

Conversation

@khakhlyuk
Copy link
Contributor

What changes were proposed in this pull request?

Following #53241 I am adding sql states for a set of PySparkException error classes.

I chose the following sql states mappings:

  • 42K09 - "Data type mismatch"
    • a long list of error classes like NOT_LIST_OR_STR_OR_TUPLE, quite self-explanatory
  • 0A000 - "feature not supported"
    • ATTRIBUTE_NOT_SUPPORTED, CLASSIC_OPERATION_NOT_SUPPORTED_ON_DF, JVM_ATTRIBUTE_NOT_SUPPORTED, NOT_IMPLEMENTED, ONLY_SUPPORTED_WITH_SPARK_CONNECT
  • 08003 - "connection does not exist"
    • NO_ACTIVE_SESSION, NO_ACTIVE_OR_DEFAULT_SESSION
    • The 08003 SQL state is a standard ANSI/ISO state meaning "connection does not exist". A Spark session is conceptually similar to a database connection. If there is no active session, a connection doesn't exist.
  • 08002 - "connection name in use" (spark) / "connection already exists" (IBM DB2)
    • SESSION_ALREADY_EXIST
    • "Connection already exists" would map perfectly to the error class, "connection name in use" sounds a bit confusing but semantically 08002 is still the best sql state for SESSION_ALREADY_EXIST, especially if we choose 08003 for NO_ACTIVE_SESSION.

Why are the changes needed?

sql states allow to better categorize and understand exceptions coming from pyspark

Does this PR introduce any user-facing change?

Users will be able to read the newly populated sql states using e.getSqlState()

How was this patch tested?

NA, mapping changes.

Was this patch authored or co-authored using generative AI tooling?

No

Copy link
Contributor

@allisonwang-db allisonwang-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gaogaotiantian
gaogaotiantian approved these changes Dec 10, 2025
@HyukjinKwon
Copy link
Member

Merged to master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants