Occasional leak of Thread/Channel instances and threads blocking indefinately

Describe the bug

We noticed that occasionally we have threads stuck forever with the sack that point to the RabbitMQ channel class.
The stack is below. Generally there are several notes about this:

1. The the RPC timeout is not applied to the enqueueAsyncRpc part of the processing only to the IO part
   This makes it possible for the thread to wait forever for _activeRpc to clear which in this case never happens.
   i.e. the code below can loop forever disregarding the rpc timeout.

        while(this._activeRpc != null) {
            try {
                this._channelLockCondition.await();
            } catch (InterruptedException var7) {
                var2 = true;
            }
        }
   

   I guess the same rpc timeout note goes for any "lock" calls
   this._channelLock.lock()
   A thread can also get stuck there forever.

2. The code was recently refactored to use locks instead of the synchronized sections
   It's not clear how that should work in the multi-threaded environment (java memory model) with members like _activeRpc.
   They are not declared as volatile/atomic. How would memory barrier be enforced for multi-threaded access to the
   member?

3. Also the semantic of the at ChannelN.asyncCompletableRpc is async, so it should never block,
   but in fact it can and does block.

========================== Threads get stuck in this state indefinitely ================

java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@17.0.9/Native Method)

• parking to wait for <0x00000007a5c699d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  at java.util.concurrent.locks.LockSupport.park(java.base@17.0.9/LockSupport.java:341)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@17.0.9/AbstractQueuedSynchronizer.java:506)
  at java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@17.0.9/ForkJoinPool.java:3465)
  at java.util.concurrent.ForkJoinPool.managedBlock(java.base@17.0.9/ForkJoinPool.java:3436)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@17.0.9/AbstractQueuedSynchronizer.java:1623)
  at com.rabbitmq.client.impl.AMQChannel.doEnqueueRpc
  at com.rabbitmq.client.impl.AMQChannel.enqueueAsyncRpc
  at com.rabbitmq.client.impl.AMQChannel.quiescingAsyncRpc
  at com.rabbitmq.client.impl.AMQChannel.asyncRpc
  at com.rabbitmq.client.impl.AMQChannel.privateAsyncRpc
  at com.rabbitmq.client.impl.AMQChannel.exnWrappingAsyncRpc
  at com.rabbitmq.client.impl.ChannelN.asyncCompletableRpc
  at reactor.rabbitmq.Sender.lambda$declareExchange$16
  ...

Reproduction steps

It's not clear what is triggering the condition.

Expected behavior

Any invocation should respect the rpc timeout.

Additional context

The code is using 5.20.0 version of the java client.

Start at AMQChannel.doEnqueueRpc and trace the wait and timeout behavior through enqueueAsyncRpc, quiescingAsyncRpc, asyncRpc, and privateAsyncRpc. Then inspect ChannelN.asyncCompletableRpc and the reactor Sender declaration path. Done means invocations respect the RPC timeout instead of waiting indefinitely, with coverage for the reported blocked state.