-
Notifications
You must be signed in to change notification settings - Fork 34
Description
While testing on https://github.com/trinodb/trino I have seen on CI several times this kind of exception:
Caused by: com.amazon.redshift.util.RedshiftException: ERROR: could not complete because of conflict with concurrent transaction
at com.amazon.redshift.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2648)
...
at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1629)
at io.trino.plugin.jdbc.BaseJdbcClient.finishInsertTable(BaseJdbcClient.java:1195)
... 27 more
Suppressed: java.lang.RuntimeException: Query: INSERT INTO "***"."test_schema"."test_row_updateesa4qfftu2" ("nationkey", "name", "regionkey", "comment") SELECT "nationkey", "name", "regionkey", "comment" FROM "***"."test_schema"."tmp_trino_97a22eb1_bdfe6d29" temp_table WHERE EXISTS (SELECT 1 FROM "***"."test_schema"."tmp_trino_97a22eb1_5a157fbd" page_sink_table WHERE page_sink_table.trino_page_sink_id = temp_table.trino_page_sink_id)
at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1632)
... 28 more
While searching the web, I've stumbled as well on dbt-labs/dbt-core#2967 which basically reports a problem very similar to the one that I'm experiencing.
tldr; The Redshift exception "could not complete because of conflict with concurrent transaction" seems to happen (on the server-side) even in situations where multiple concurrent operations only do DML operations by selecting from the same source table.
My question here is applies rather generically - can such an exception as the one listed in the above stacktrace be considered "retriable" ?
If yes, I plan to apply retries for generic operations executed on Redshift through the trino-redshift connector.
I'm considering to add this kind of handling to be focused on DML operations.
Code taken from a chatgpt output (obviously to be taken with a grain of salt):
private boolean isRetriable(RedshiftException e) {
String msg = e.getMessage();
if (msg == null) return false;
// Redshift serialization-conflict messages
return msg.contains("conflict with concurrent transaction")
|| msg.contains("Serializable isolation violation")
|| msg.contains("Serialization error")
|| msg.contains("1023"); // Redshift error code for these conflicts
}