aws / aws/amazon-redshift-jdbc-driver
Are the DML & DDL statements failing due concurrency conflicts retriable?
- Dominant language
- Java
- Stars
- 71
- Forks
- 42
- PR merge metrics
- No merged PRs in 30d
Description
While testing on https://github.com/trinodb/trino I have seen on CI several times this kind of exception:
```
Caused by: com.amazon.redshift.util.RedshiftException: ERROR: could not complete because of conflict with concurrent transaction
at com.amazon.redshift.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2648)
...
at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1629)
at io.trino.plugin.jdbc.BaseJdbcClient.finishInsertTable(BaseJdbcClient.java:1195)
... 27 more
Suppressed: java.lang.RuntimeException: Query: INSERT INTO "***"."test_schema"."test_row_updateesa4qfftu2" ("nationkey", "name", "regionkey", "comment") SELECT "nationkey", "name", "regionkey", "comment" FROM "***"."test_schema"."tmp_trino_97a22eb1_bdfe6d29" temp_table WHERE EXISTS (SELECT 1 FROM "***"."test_schema"."tmp_trino_97a22eb1_5a157fbd" page_sink_table WHERE page_sink_table.trino_page_sink_id = temp_table.trino_page_sink_id)
at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1632)
... 28 more
```
While searching the web, I've stumbled as well on https://github.com/dbt-labs/dbt-core/issues/2967 which basically reports a problem very similar to the one that I'm experiencing.
tldr; The Redshift exception "could not complete because of conflict with concurrent transaction" seems to happen (on the server-side) even in situations where multiple concurrent operations only do DML operations by selecting from the same source table.
My question here is applies rather generically - can such an exception as the one listed in the above stacktrace be considered "**retriable**" ?
~~If yes, I plan to apply retries for generic operations executed on Redshift through the `trino-redshift` connector.
I'm considering to add this kind of handling to be focused on DML operations.~~
Code taken from a chatgpt output (obviously to be taken with a grain of salt):
```
private boolean isRetriable(RedshiftException e) {
String msg = e.getMessage();
if (msg == null) return false;
// Redshift serialization-conflict messages
return msg.contains("conflict with concurrent transaction")
|| msg.contains("Serializable isolation violation")
|| msg.contains("Serialization error")
|| msg.contains("1023"); // Redshift error code for these conflicts
}
```
Contributor guide
Research direction
Start with RedshiftException and the QueryExecutorImpl and BaseJdbcClient frames in the supplied stack trace, then compare the concurrent-transaction error with the proposed error-message checks. Done means establishing and documenting whether this conflict is retriable and whether the JDBC driver needs a defined handling change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100