iceberg-kafka-connect zombie coordinator thread when writing to S3 fails

### Apache Iceberg version

1.10.1 (latest release)

### Query engine

None

### Please describe the bug 🐞

We noticed an issue with iceberg-kafka-connect when writes to S3 fail due to network issues. The failed writes seem to cause a zombie coordinator thread which is not properly shut down and continues to operate on a catalog reference that was shut down before.

Logs of the root cause (happening once):
```
[Worker-091d01c58937b73e7] [2026-04-02 14:17:54,989] INFO [archiver|task-0] Sending event of type: START_COMMIT (org.apache.iceberg.connect.channel.Channel:88)
[Worker-091d01c58937b73e7] [2026-04-02 14:17:55,066] INFO [archiver|task-0] Commit a1f40937-c8a2-4361-a9dd-30f4a295bd79 initiated (org.apache.iceberg.connect.channel.Coordinator:116)
[Worker-091d01c58937b73e7] [2026-04-02 14:17:57,311] ERROR [archiver|task-0] WorkerSinkTask{id=archiver-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. Error: java.io.IOException: software.amazon.awssdk.core.exception.SdkClientException: Received an UnknownHostException when attempting to interact with a service. See cause for the exact endpoint that is failing to resolve. If this is happening on an endpoint that previously worked, there may be a network connectivity issue or your DNS cache could be storing endpoints for too long. (org.apache.kafka.connect.runtime.WorkerSinkTask:630)
[Worker-091d01c58937b73e7] java.io.UncheckedIOException: java.io.IOException: software.amazon.awssdk.core.exception.SdkClientException: Received an UnknownHostException when attempting to interact with a service. See cause for the exact endpoint that is failing to resolve. If this is happening on an endpoint that previously worked, there may be a network connectivity issue or your DNS cache could be storing endpoints for too long.
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.data.IcebergWriter.flush(IcebergWriter.java:105)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.data.IcebergWriter.complete(IcebergWriter.java:118)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.data.SinkWriter.lambda$completeWrite$0(SinkWriter.java:57)
[Worker-091d01c58937b73e7] at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)
[Worker-091d01c58937b73e7] at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1779)
[Worker-091d01c58937b73e7] at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
[Worker-091d01c58937b73e7] at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
[Worker-091d01c58937b73e7] at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
[Worker-091d01c58937b73e7] at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
[Worker-091d01c58937b73e7] at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.data.SinkWriter.completeWrite(SinkWriter.java:58)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.channel.Worker.receive(Worker.java:75)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.channel.Channel.lambda$consumeAvailable$2(Channel.java:132)
[Worker-091d01c58937b73e7] at java.base/java.lang.Iterable.forEach(Iterable.java:75)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.channel.Channel.consumeAvailable(Channel.java:122)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.channel.Worker.process(Worker.java:65)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.channel.CommitterImpl.processControlEvents(CommitterImpl.java:173)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.channel.CommitterImpl.save(CommitterImpl.java:165)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.IcebergSinkTask.put(IcebergSinkTask.java:83)
[Worker-091d01c58937b73e7] at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:601)
[Worker-091d01c58937b73e7] at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:350)
[Worker-091d01c58937b73e7] at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:250)
[Worker-091d01c58937b73e7] at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:219)
[Worker-091d01c58937b73e7] at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:204)
[Worker-091d01c58937b73e7] at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:259)
[Worker-091d01c58937b73e7] at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:237)
[Worker-091d01c58937b73e7] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
[Worker-091d01c58937b73e7] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[Worker-091d01c58937b73e7] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[Worker-091d01c58937b73e7] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[Worker-091d01c58937b73e7] at java.base/java.lang.Thread.run(Thread.java:840)
[Worker-091d01c58937b73e7] Caused by: java.io.IOException: software.amazon.awssdk.core.exception.SdkClientException: Received an UnknownHostException when attempting to interact with a service. See cause for the exact endpoint that is failing to resolve. If this is happening on an endpoint that previously worked, there may be a network connectivity issue or your DNS cache could be storing endpoints for too long.
[Worker-091d01c58937b73e7] at org.apache.parquet.io.DelegatingPositionOutputStream.close(DelegatingPositionOutputStream.java:41)
[Worker-091d01c58937b73e7] at org.apache.parquet.hadoop.ParquetFileWriter.close(ParquetFileWriter.java:1830)
[Worker-091d01c58937b73e7] at org.apache.parquet.hadoop.ParquetFileWriter.end(ParquetFileWriter.java:1816)
[Worker-091d01c58937b73e7] at org.apache.iceberg.parquet.ParquetWriter.close(ParquetWriter.java:261)
[Worker-091d01c58937b73e7] at org.apache.iceberg.io.DataWriter.close(DataWriter.java:82)
[Worker-091d01c58937b73e7] at org.apache.iceberg.io.BaseTaskWriter$BaseRollingWriter.closeCurrent(BaseTaskWriter.java:348)
[Worker-091d01c58937b73e7] at org.apache.iceberg.io.BaseTaskWriter$BaseRollingWriter.close(BaseTaskWriter.java:380)
[Worker-091d01c58937b73e7] at org.apache.iceberg.io.PartitionedFanoutWriter.close(PartitionedFanoutWriter.java:70)
[Worker-091d01c58937b73e7] at org.apache.iceberg.io.BaseTaskWriter.complete(BaseTaskWriter.java:103)
[Worker-091d01c58937b73e7] at org.apache.iceberg.connect.data.IcebergWriter.flush(IcebergWriter.java:103)
[Worker-091d01c58937b73e7] ... 30 more
[Worker-091d01c58937b73e7] Caused by: software.amazon.awssdk.core.exception.SdkClientException: Received an UnknownHostException when attempting to interact with a service. See cause for the exact endpoint that is failing to resolve. If this is happening on an endpoint that previously worked, there may be a network connectivity issue or your DNS cache could be storing endpoints for too long.
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:130)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.awscore.interceptor.HelpfulUnknownHostExceptionInterceptor.modifyException(HelpfulUnknownHostExceptionInterceptor.java:59)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.modifyException(ExecutionInterceptorChain.java:181)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.ExceptionReportingUtils.runModifyException(ExceptionReportingUtils.java:54)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.ExceptionReportingUtils.reportFailureToInterceptors(ExceptionReportingUtils.java:38)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:39)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:210)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.services.s3.DefaultS3Client.putObject(DefaultS3Client.java:11883)
[Worker-091d01c58937b73e7] at org.apache.iceberg.aws.s3.S3OutputStream.completeUploads(S3OutputStream.java:443)
[Worker-091d01c58937b73e7] at org.apache.iceberg.aws.s3.S3OutputStream.close(S3OutputStream.java:269)
[Worker-091d01c58937b73e7] at org.apache.iceberg.aws.s3.S3OutputStream.close(S3OutputStream.java:255)
[Worker-091d01c58937b73e7] at org.apache.parquet.io.DelegatingPositionOutputStream.close(DelegatingPositionOutputStream.java:40)
[Worker-091d01c58937b73e7] ... 39 more
[Worker-091d01c58937b73e7] Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: archiver-bucket-12345.s3.eu-west-1.amazonaws.com (SDK Attempt Count: 6)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:130)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:95)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.retryPolicyDisallowedRetryException(RetryableStageHelper.java:168)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:73)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:53)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:35)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:82)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:62)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:43)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
[Worker-091d01c58937b73e7] ... 53 more
[Worker-091d01c58937b73e7] Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 1 failure: Unable to execute HTTP request: archiver-bucket-12345.s3.eu-west-1.amazonaws.com: No address associated with hostname
[Worker-091d01c58937b73e7] Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 2 failure: Unable to execute HTTP request: archiver-bucket-12345.s3.eu-west-1.amazonaws.com
[Worker-091d01c58937b73e7] Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 3 failure: Unable to execute HTTP request: archiver-bucket-12345.s3.eu-west-1.amazonaws.com
[Worker-091d01c58937b73e7] Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 4 failure: Unable to execute HTTP request: archiver-bucket-12345.s3.eu-west-1.amazonaws.com
[Worker-091d01c58937b73e7] Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 5 failure: Unable to execute HTTP request: archiver-bucket-12345.s3.eu-west-1.amazonaws.com
[Worker-091d01c58937b73e7] Caused by: java.net.UnknownHostException: archiver-bucket-12345.s3.eu-west-1.amazonaws.com
[Worker-091d01c58937b73e7] at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
[Worker-091d01c58937b73e7] at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1533)
[Worker-091d01c58937b73e7] at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1385)
[Worker-091d01c58937b73e7] at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1306)
[Worker-091d01c58937b73e7] at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
[Worker-091d01c58937b73e7] at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
[Worker-091d01c58937b73e7] at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.http.apache.internal.conn.ClientConnectionManagerFactory$DelegatingHttpClientConnectionManager.connect(ClientConnectionManagerFactory.java:86)
[Worker-091d01c58937b73e7] at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
[Worker-091d01c58937b73e7] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
[Worker-091d01c58937b73e7] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
[Worker-091d01c58937b73e7] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[Worker-091d01c58937b73e7] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[Worker-091d01c58937b73e7] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.http.apache.internal.impl.ApacheSdkHttpClient.execute(ApacheSdkHttpClient.java:72)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.http.apache.ApacheHttpClient.execute(ApacheHttpClient.java:261)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.http.apache.ApacheHttpClient.access$600(ApacheHttpClient.java:106)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:238)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:235)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:103)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.executeHttpRequest(MakeHttpRequestStage.java:88)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:64)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:46)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:74)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:43)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:79)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:41)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.executeRequest(RetryableStage.java:93)
[Worker-091d01c58937b73e7] at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:56)
[Worker-091d01c58937b73e7] ... 65 more
```

Log of the zombie coordinator errors (happening continuously):
```
[Worker-00751b5035a65aa00] [2026-04-11 07:04:56,160] INFO [archiver|task-1] Commit timeout reached. Commit ID: 35b17148-1c67-4f9e-a5b8-9f0c4ccdc7f9 (org.apache.iceberg.connect.channel.CommitState:106)
[Worker-00751b5035a65aa00] [2026-04-11 07:04:56,162] INFO [archiver|task-1] Loading custom FileIO implementation: org.apache.iceberg.aws.s3.S3FileIO (org.apache.iceberg.CatalogUtil:382)
[Worker-00751b5035a65aa00] [2026-04-11 07:04:56,172] WARN [archiver|task-1] Commit failed, will try again next cycle (org.apache.iceberg.connect.channel.Coordinator:146)
[Worker-00751b5035a65aa00] java.lang.IllegalStateException: Connection pool shut down
[Worker-00751b5035a65aa00] at org.apache.http.util.Asserts.check(Asserts.java:34)
[Worker-00751b5035a65aa00] at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:269)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.http.apache.internal.conn.ClientConnectionManagerFactory$DelegatingHttpClientConnectionManager.requestConnection(ClientConnectionManagerFactory.java:75)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.http.apache.internal.conn.ClientConnectionManagerFactory$InstrumentedHttpClientConnectionManager.requestConnection(ClientConnectionManagerFactory.java:57)
[Worker-00751b5035a65aa00] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:176)
[Worker-00751b5035a65aa00] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
[Worker-00751b5035a65aa00] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[Worker-00751b5035a65aa00] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[Worker-00751b5035a65aa00] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.http.apache.internal.impl.ApacheSdkHttpClient.execute(ApacheSdkHttpClient.java:72)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.http.apache.ApacheHttpClient.execute(ApacheHttpClient.java:261)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.http.apache.ApacheHttpClient.access$600(ApacheHttpClient.java:106)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:238)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:235)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:103)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.executeHttpRequest(MakeHttpRequestStage.java:88)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:64)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:46)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:74)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:43)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:79)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:41)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.executeRequest(RetryableStage.java:93)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:56)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:53)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:35)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:82)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:62)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:43)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:210)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
[Worker-00751b5035a65aa00] at software.amazon.awssdk.services.glue.DefaultGlueClient.getTable(DefaultGlueClient.java:31534)
[Worker-00751b5035a65aa00] at org.apache.iceberg.aws.glue.GlueTableOperations.getGlueTable(GlueTableOperations.java:279)
[Worker-00751b5035a65aa00] at org.apache.iceberg.aws.glue.GlueTableOperations.doRefresh(GlueTableOperations.java:128)
[Worker-00751b5035a65aa00] at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:88)
[Worker-00751b5035a65aa00] at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:71)
[Worker-00751b5035a65aa00] at org.apache.iceberg.BaseMetastoreCatalog.loadTable(BaseMetastoreCatalog.java:49)
[Worker-00751b5035a65aa00] at org.apache.iceberg.connect.channel.Coordinator.commitToTable(Coordinator.java:199)
[Worker-00751b5035a65aa00] at org.apache.iceberg.connect.channel.Coordinator.lambda$doCommit$1(Coordinator.java:163)
[Worker-00751b5035a65aa00] at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
[Worker-00751b5035a65aa00] at org.apache.iceberg.util.Tasks$Builder$1.run(Tasks.java:315)
[Worker-00751b5035a65aa00] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
[Worker-00751b5035a65aa00] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[Worker-00751b5035a65aa00] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[Worker-00751b5035a65aa00] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[Worker-00751b5035a65aa00] at java.base/java.lang.Thread.run(Thread.java:840)
```

Worker ids don't match because I took the logs from two different runs.
This is happening rather sporadically (~weeks between incidents).
Our setup is: AWS MSK Connect with iceberg-kafka-connect plugin installed

### Willingness to contribute

- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Start with IcebergWriter.java and the coordinator path through SinkWriter.java, Worker.java, Channel.java, CommitterImpl.java, and IcebergSinkTask.java. Trace cleanup after the S3 write failure, then reproduce the failure and verify that the coordinator thread no longer operates on a shut-down catalog; add or update a regression test if the connector tests cover this path.

Written by the indexing model from the issue text.