[Bug] Proxy telemetry dispatch failure leaves relay futures pending until timeout
- Dominant language
- Java
- Stars
- 22.6k
- Forks
- 12k
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 27
Description
### Before Creating the Bug Report
- [x] I have confirmed this problem is not already covered by an existing issue or open pull request.
### Runtime platform environment
RocketMQ Proxy with gRPC telemetry clients.
### RocketMQ version
develop branch
### JDK Version
N/A
### Describe the Bug
`GrpcClientChannel` registers response futures before writing telemetry commands for remote diagnostic/admin operations. If the telemetry observer is already unavailable, or if `observer.onNext()` fails before the command is delivered, `writeTelemetryCommand()` only logs and returns.
The registered nonce remains in `GrpcChannelManager.resultNonceFutureMap` until the periodic timeout scanner eventually completes it with `SYSTEM_BUSY`.
Affected flows include:
- `processGetConsumerRunningInfo()` when thread-stack collection is requested
- `processConsumeMessageDirectly()`
### Steps to Reproduce
1. Register a gRPC client channel.
2. Trigger a diagnostic command such as consumer running info with `jstackEnable=true`.
3. Make the telemetry observer unavailable or make `observer.onNext()` throw.
4. Observe that the response future remains pending until the relay timeout scan runs.
### What Did You Expect to See?
When telemetry dispatch fails locally, the registered nonce should be removed and the waiting response future should complete immediately with an error result.
### What Did You See Instead?
The response future stays pending until the periodic timeout cleanup fires.
### Additional Context
This is related to the Proxy Admin/runtime diagnostics work: admin calls should fail fast when the gRPC telemetry stream is not writable, instead of waiting for the full relay timeout.
Contributor guide
Research direction
Start with GrpcClientChannel.writeTelemetryCommand() and resultNonceFutureMap, then trace the processGetConsumerRunningInfo() and processConsumeMessageDirectly() flows described in the issue. Reproduce a failed observer.onNext() dispatch and verify that the nonce is removed and the waiting response completes immediately with an error instead of waiting for timeout cleanup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grpc, java
- Domain
- api, backend, observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100