aws / aws/amazon-redshift-jdbc-driver

Are the DML & DDL statements failing due concurrency conflicts retriable?

Open
#147 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
71
Forks
42
PR merge metrics
No merged PRs in 30d

Description

While testing on https://github.com/trinodb/trino I have seen on CI several times this kind of exception:

```
Caused by: com.amazon.redshift.util.RedshiftException: ERROR: could not complete because of conflict with concurrent transaction
at com.amazon.redshift.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2648)
...
at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1629)
at io.trino.plugin.jdbc.BaseJdbcClient.finishInsertTable(BaseJdbcClient.java:1195)
... 27 more
Suppressed: java.lang.RuntimeException: Query: INSERT INTO "***"."test_schema"."test_row_updateesa4qfftu2" ("nationkey", "name", "regionkey", "comment") SELECT "nationkey", "name", "regionkey", "comment" FROM "***"."test_schema"."tmp_trino_97a22eb1_bdfe6d29" temp_table WHERE EXISTS (SELECT 1 FROM "***"."test_schema"."tmp_trino_97a22eb1_5a157fbd" page_sink_table WHERE page_sink_table.trino_page_sink_id = temp_table.trino_page_sink_id)
at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1632)
... 28 more
```

While searching the web, I've stumbled as well on https://github.com/dbt-labs/dbt-core/issues/2967 which basically reports a problem very similar to the one that I'm experiencing.

tldr; The Redshift exception "could not complete because of conflict with concurrent transaction" seems to happen (on the server-side) even in situations where multiple concurrent operations only do DML operations by selecting from the same source table.

My question here is applies rather generically - can such an exception as the one listed in the above stacktrace be considered "**retriable**" ?
~~If yes, I plan to apply retries for generic operations executed on Redshift through the `trino-redshift` connector.
I'm considering to add this kind of handling to be focused on DML operations.~~

Code taken from a chatgpt output (obviously to be taken with a grain of salt):

```
private boolean isRetriable(RedshiftException e) {
String msg = e.getMessage();
if (msg == null) return false;

// Redshift serialization-conflict messages
return msg.contains("conflict with concurrent transaction")
|| msg.contains("Serializable isolation violation")
|| msg.contains("Serialization error")
|| msg.contains("1023"); // Redshift error code for these conflicts
}
```

Contributor guide

Open the contributing guide

Research direction

Start with RedshiftException and the QueryExecutorImpl and BaseJdbcClient frames in the supplied stack trace, then compare the concurrent-transaction error with the proposed error-message checks. Done means establishing and documenting whether this conflict is retriable and whether the JDBC driver needs a defined handling change.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.