apache / apache/iceberg

MySQL JDBC catalog does not recover from a dropped connection

Open
#17,829 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.11.0 (latest release)

### Query engine

Other

### Please describe the bug 🐞

When a MySQL-backed `JdbcCatalog` connection is dropped by the server (for example an idle connection closed at `wait_timeout`), the pool does not recover it:

```
UncheckedSQLException: Failed to execute exists query: SELECT ... FROM iceberg_tables ...
Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
Caused by: java.net.SocketException: Broken pipe
```

The failure does not clear on the next call. `ClientPoolImpl.release` returns the client with `addFirst`, so the dead connection goes back to the head of the pool and is handed to the next caller.

`JdbcClientPool` already reconnects on connection failures, but only for `SQLTransientException` and the SQLSTATEs in `COMMON_RETRYABLE_CONNECTION_SQL_STATES`. MySQL's `CommunicationsException` extends `SQLRecoverableException` — a sibling of `SQLTransientException`, not a subtype — and reports SQLSTATE `08S01`, which is not in that set. Neither check matches, so no reconnect is attempted.

Same symptom as #14317.

Proposed fix: treat `SQLRecoverableException` as retryable. The JDBC contract for that type is that recovery requires closing the connection and obtaining a new one, which is what `JdbcClientPool.reconnect` does. Adding `08S01` to the default SQLSTATEs would fix MySQL only.

On existing releases, `retryable_status_codes=08S01` works as a workaround.

PR: #17828

Contributor guide

Open the contributing guide

Research direction

Start with JdbcClientPool's retry checks and reconnect method, then inspect ClientPoolImpl.release to understand how failed clients return to the pool. Review PR #17828 and existing pool tests; done means a dropped MySQL connection is replaced and the next catalog call succeeds without the CommunicationsException recurring.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.