Skip to content

Are the DML & DDL statements failing due concurrency conflicts retriable? #147

@findinpath

Description

@findinpath

While testing on https://github.com/trinodb/trino I have seen on CI several times this kind of exception:

Caused by: com.amazon.redshift.util.RedshiftException: ERROR: could not complete because of conflict with concurrent transaction
	at com.amazon.redshift.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2648)
...
	at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1629)
	at io.trino.plugin.jdbc.BaseJdbcClient.finishInsertTable(BaseJdbcClient.java:1195)
	... 27 more
	Suppressed: java.lang.RuntimeException: Query: INSERT INTO "***"."test_schema"."test_row_updateesa4qfftu2" ("nationkey", "name", "regionkey", "comment") SELECT "nationkey", "name", "regionkey", "comment" FROM "***"."test_schema"."tmp_trino_97a22eb1_bdfe6d29" temp_table WHERE EXISTS (SELECT 1 FROM "***"."test_schema"."tmp_trino_97a22eb1_5a157fbd" page_sink_table WHERE page_sink_table.trino_page_sink_id = temp_table.trino_page_sink_id)
		at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1632)
		... 28 more

While searching the web, I've stumbled as well on dbt-labs/dbt-core#2967 which basically reports a problem very similar to the one that I'm experiencing.

tldr; The Redshift exception "could not complete because of conflict with concurrent transaction" seems to happen (on the server-side) even in situations where multiple concurrent operations only do DML operations by selecting from the same source table.

My question here is applies rather generically - can such an exception as the one listed in the above stacktrace be considered "retriable" ?
If yes, I plan to apply retries for generic operations executed on Redshift through the trino-redshift connector.
I'm considering to add this kind of handling to be focused on DML operations.

Code taken from a chatgpt output (obviously to be taken with a grain of salt):

private boolean isRetriable(RedshiftException e) {
    String msg = e.getMessage();
    if (msg == null) return false;

    // Redshift serialization-conflict messages
    return msg.contains("conflict with concurrent transaction")
        || msg.contains("Serializable isolation violation")
        || msg.contains("Serialization error")
        || msg.contains("1023"); // Redshift error code for these conflicts
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions