[SUPPORT] Hudi Application getting stuck when Async cleaner is spawned

**_Tips before filing an issue_**

- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?

- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.

- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.

**Describe the problem you faced**

After some failed commits, Hudi application thread-pool is stuck while acquiring the lock without any progress.

A clear and concise description of the problem.

When Hudi application is stopped with some error or ungracefully for some time, when it is restarted it is getting stuck with the below logs without any progress for ~115 minutes.

**To Reproduce**

Steps to reproduce the behavior:

1. Start Hudi application and run it for sometime
2. Kill application in the middle of the commit and then restart again
3. Use InProcessLockProvider and enable async cleaning and compaction.

**Expected behavior**
Pipeline should not stuck and run without error.
A clear and concise description of what you expected to happen.

**Environment Description**

* Hudi version : 0.12.1

* Spark version : 3.2.2

* Hive version : 2.3.5

* Hadoop version : 2.7.7

* Storage (HDFS/S3/GCS..) : GCS

* Running on Docker? (yes/no) : yes

**Additional context**

config:

# cleaner config
hoodie.cleaner.policy=KEEP_LATEST_COMMITS
hoodie.clean.automatic=true
hoodie.clean.async=true
hoodie.cleaner.commits.retained=5
hoodie.keep.min.commits=10
#compaction config
hoodie.datasource.compaction.async.enable=true
hoodie.parquet.small.file.limit=1048576
hoodie.compaction.target.io=50
hoodie.metadata.metrics.enable=true
#metadata table
hoodie.metadata.index.bloom.filter.enable=false
hoodie.metadata.index.column.stats.enable=false
#Single writer with async table services
hoodie.write.concurrency.mode=optimistic_concurrency_control
hoodie.cleaner.policy.failed.writes=LAZY
hoodie.write.lock.provider=org.apache.hudi.client.transaction.lock.InProcessLockProvider
hoodie.write.lock.wait_time_ms=300000

Add any other context about the problem here.

**Stacktrace**

```
22/12/02 08:39:31 INFO pool-31-thread-1 BaseHoodieWriteClient: Async cleaner has been spawned. Waiting for it to finish
22/12/02 08:39:31 INFO pool-31-thread-1 AsyncCleanerService: Waiting for async clean service to finish
22/12/02 08:39:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:39:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:39:55 INFO dispatcher-BlockManagerMaster BlockManagerInfo: Removed broadcast_48_piece0 on hudi-prod-data-mor-b17eab84d1f75c1f-driver-svc.spark-jobs.svc:7079 in memory (size: 36.2 KiB, free: 1663.2 MiB)
22/12/02 08:39:55 INFO dispatcher-BlockManagerMaster BlockManagerInfo: Removed broadcast_48_piece0 on 10.207.18.169:46689 in memory (size: 36.2 KiB, free: 1048.4 MiB)
22/12/02 08:39:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:40:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:40:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:40:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:40:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:40:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:40:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:41:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:41:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:41:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:41:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:41:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:41:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:42:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:42:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:42:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:42:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:42:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:42:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:43:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:43:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:43:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:43:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:43:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:43:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:44:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:44:12 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 FAILED_TO_ACQUIRE in-process lock.
22/12/02 08:44:12 INFO pool-36-thread-1 LockManager: Retrying to acquire lock...
22/12/02 08:44:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:44:22 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 ACQUIRING in-process lock.
22/12/02 08:44:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:44:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:44:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:44:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:45:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:45:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:45:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:45:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:45:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:45:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:46:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:46:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:46:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:46:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:46:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:46:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:47:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:47:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:47:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:47:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:47:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:47:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:48:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:48:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:48:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:48:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:48:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:48:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:49:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:49:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:49:22 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 FAILED_TO_ACQUIRE in-process lock.
22/12/02 08:49:22 INFO pool-36-thread-1 LockManager: Retrying to acquire lock...
22/12/02 08:49:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:49:32 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 ACQUIRING in-process lock.
22/12/02 08:49:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:49:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:49:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:50:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:50:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:50:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:50:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:50:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:50:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:51:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:51:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:51:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:51:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:51:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:51:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:52:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:52:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:52:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:52:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:52:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:52:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:53:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:53:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:53:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:53:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:53:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:53:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:54:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:54:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:54:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:54:32 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 FAILED_TO_ACQUIRE in-process lock.
22/12/02 08:54:32 INFO pool-36-thread-1 LockManager: Retrying to acquire lock...
22/12/02 08:54:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:54:42 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 ACQUIRING in-process lock.
22/12/02 08:54:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:54:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:55:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:55:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:55:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:55:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:55:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:55:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:56:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:56:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:56:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:56:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:56:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:56:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:57:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:57:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:57:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:57:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:57:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:57:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:58:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:58:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:58:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:58:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:58:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:58:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:59:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:59:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:59:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:59:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:59:42 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 FAILED_TO_ACQUIRE in-process lock.
22/12/02 08:59:42 INFO pool-36-thread-1 LockManager: Retrying to acquire lock...
22/12/02 08:59:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 08:59:52 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 ACQUIRING in-process lock.
22/12/02 08:59:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:00:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:00:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:00:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:00:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:00:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:00:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:01:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:01:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:01:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:01:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:01:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:01:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:02:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:02:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:02:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:02:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:02:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:02:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:03:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:03:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:03:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:03:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:03:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:03:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:04:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:04:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:04:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:04:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:04:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:04:52 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 FAILED_TO_ACQUIRE in-process lock.
22/12/02 09:04:52 INFO pool-36-thread-1 LockManager: Retrying to acquire lock...
22/12/02 09:04:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:05:02 INFO pool-36-thread-1 InProcessLockProvider: Thread pool-36-thread-1 ACQUIRING in-process lock.
22/12/02 09:05:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:05:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:05:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:05:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:05:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:05:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:06:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:06:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:06:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:06:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:06:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:06:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:07:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:07:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:07:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:07:37 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:07:47 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:07:57 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:08:07 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:08:17 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
22/12/02 09:08:27 INFO async_compact_thread-thread-1 HoodieAsyncService: Waiting for next instant up to 10 seconds
```

Start by reproducing the restart scenario with InProcessLockProvider, asynchronous cleaning and compaction, using the listed Hudi, Spark, Hive, Hadoop and GCS environment. Read the stack-trace entry points in BaseHoodieWriteClient, AsyncCleanerService, HoodieAsyncService, InProcessLockProvider and LockManager to trace the lock wait. Done means the pipeline progresses after an interrupted commit and restart instead of remaining stuck in repeated lock acquisition and async-service waits.

Written by the indexing model from the issue text.