apache / apache/rocketmq

[Bug] Remoting sub-servers do not honor graceful shutdown

Open
#11,151 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
22.6k
Forks
12k
Avg merge
3d 1h
Merged PRs (30d)
27

Description

### Runtime platform environment

macOS aarch64, JDK 11.

### RocketMQ version

`develop`, commit `d38f81e787`.

### Describe the Bug

Sub-servers created by `NettyRemotingServer.newRemotingServer(port)` do not honor the parent's graceful-shutdown configuration. `SubRemotingServer.shutdown()` sets its shutdown flag and immediately closes its listening channel. During the configured grace period, new connections therefore fail instead of receiving `GO_AWAY` responses for supported client versions.

Calling only the parent's `shutdown()` has another gap: it marks only the parent as shutting down, so sub-servers can continue processing requests until the shared event loops shut down. Concurrent shutdown calls also need coordination: a repeated call must not bypass the first call's grace period and close the listener or shared resources early.

### Steps to Reproduce

1. Start a `NettyRemotingServer` with `enableShutdownGracefully=true` and `shutdownWaitTimeSeconds=5`.
2. Create and start sub-servers on other ports, and register a request processor on each server.
3. Establish a connection and send a request using a protocol version newer than `V5_3_1`.
4. Call a sub-server's `shutdown()` on another thread. During the grace period, send another request over the existing connection and try a fresh connection to that port.
5. Separately, call only the parent server's `shutdown()` and send requests to the sub-server ports during the grace period.

### What Did You Expect to See?

A standalone sub-server shutdown keeps its listener available during the configured grace period and uses the existing `GO_AWAY` handling, without shutting down other ports or shared resources.

Each sub-server measures its own grace period from when it first enters the draining state. Repeated shutdown calls must not reset that deadline. A parent shutdown marks all its sub-servers as draining and waits until its own and all child grace periods have elapsed before releasing shared resources. Main/sub shutdowns and repeated calls may run concurrently; the independent grace periods must overlap rather than being waited one after another.

### What Did You See Instead?

The standalone sub-server closes its listener immediately. Parent-only shutdown leaves sub-servers outside the draining state. A regression test on the unmodified commit times out waiting for a child to enter that state after parent shutdown starts.

Contributor guide

Open the contributing guide

Research direction

Start at NettyRemotingServer.newRemotingServer(port) and SubRemotingServer.shutdown(), then trace the existing GO_AWAY handling for clients newer than V5_3_1. Reproduce standalone and parent-only shutdown with concurrent repeated calls, and use regression coverage to verify listeners remain available during independent grace periods before shared resources are released.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.