Recovery takes a long time, how to troubleshoot
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Query engine
Iceberg KafkaConnect writing to Unity Catalog Rest API
### Question
I have a question about best practices/scaling/operating the Connector and why certain issues might be causing problems.
For example I had a connector paused for a day, due to some operations we were running. I unpaused it now, for close to one hour, it hasn't produced as single record, despite being now 2M messages behind.
In logs I only see constantly logs like:
2026-08-13 15:48:36,696 INFO [public.customer.delivery.v2-iceberg-connector|task-8] Handled event of type: START_COMMIT (org.apache.iceberg.connect.channel.Channel) [task-thread-public.customer.delivery.v2-iceberg-connector-8]
2026-08-13 15:48:36,700 INFO [public.customer.delivery.v2-unity-iceberg-connector|task-9] Handled event of type: DATA_COMPLETE (org.apache.iceberg.connect.channel.Channel) [iceberg-coord]
2026-08-13 15:48:36,700 WARN [public.customer.delivery.v2-unity-iceberg-connector|task-9] Received commit ready when no commit in progress, this can happen during recovery. Commit ID: d94b5d9e-07d6-4356-a9fb-509a754d1831 (org.apache.iceberg.connect.channel.CommitState) [iceberg-coord]
However no data is written at all...
we have got commit cadence set to 10 minutes so I would expect some records to appear every 10 minutes, but I guess there is something fundamental about Connector that I don't understand causing this.
Any tips to improve this situations/monitor to know where it will successfully create something?
Thanks!
Contributor guide
Research direction
Start with the recovery-related log sequence from org.apache.iceberg.connect.channel.Channel and CommitState, using the paused connector and its 2M-message backlog as the reproduction context. Determine where recovery progress can be observed and document the cause, monitoring guidance, and expected point at which records resume writing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100