the old server wander too long after patched by the new instance

1. deploy a cluster with new ticdc arch instances
2. patch the cdc cluster, cause all instances restarts

[2024/12/30 17:49:24.711 +08:00] [INFO] [version.go:48] ["Welcome to Change Data Capture (CDC)"] [release-version=v8.2.0-master] [git-hash=9f7f3e3ac0395dd1d10630cb4e38703469e848f8] [git-branch=read-server-code] [utc-build-time="2024-12-30 09:48:31"] [go-version="go version go1.23.4 linux/amd64"] [failpoint-build=false]
...
[2024/12/30 17:49:24.868 +08:00] [INFO] [reactor_state.go:116] ["remote capture online"] [info="{\"id\":\"67873c92-0183-4c08-bfe3-77138d9532c8\",\"address\":\"10.2.6.80:8300\",\"version\":\"v8.2.0-master\",\"git-hash\":\"65f34ab1af943cf517863e89ae5dfb87d96ec009\",\"deploy-path\":\"/root/tidb-deploy/cdc-8300/bin/cdc\",\"start-timestamp\":1735551641}"] [role=capture-manager]
[2024/12/30 17:49:24.868 +08:00] [INFO] [reactor_state.go:116] ["remote capture online"] [info="{\"id\":\"740e2769-063c-4170-889a-f5bbcd02a71f\",\"address\":\"10.2.6.80:8300\",\"version\":\"v8.2.0-master\",\"git-hash\":\"9f7f3e3ac0395dd1d10630cb4e38703469e848f8\",\"deploy-path\":\"/root/tidb-deploy/cdc-8300/bin/cdc\",\"start-timestamp\":1735552164}"] [role=capture-manager]
[2024/12/30 17:49:24.919 +08:00] [INFO] [message_center.go:167] ["Add remote target"] [local=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8] [epoch=0] [addr=10.2.6.80:8300]
[2024/12/30 17:49:24.919 +08:00] [INFO] [target.go:143] ["Create remote target"] [local=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8] [addr=10.2.6.80:8300] [localEpoch=0] [targetEpoch=0]
[2024/12/30 17:49:24.921 +08:00] [ERROR] [message_center.go:363] ["Target not found"] [error="ErrorType: Unknown, Reason: Target 67873c92-0183-4c08-bfe3-77138d9532c8 not found"]
[2024/12/30 17:49:24.921 +08:00] [ERROR] [message_center.go:363] ["Target not found"] [error="ErrorType: Unknown, Reason: Target 67873c92-0183-4c08-bfe3-77138d9532c8 not found"]
...
[2024/12/30 17:49:38.945 +08:00] [ERROR] [message_center.go:363] ["Target not found"] [error="ErrorType: Unknown, Reason: Target 67873c92-0183-4c08-bfe3-77138d9532c8 not found"]
[2024/12/30 17:49:38.945 +08:00] [ERROR] [message_center.go:363] ["Target not found"] [error="ErrorType: Unknown, Reason: Target 67873c92-0183-4c08-bfe3-77138d9532c8 not found"]
...
[2024/12/30 17:49:38.945 +08:00] [WARN] [target.go:202] ["received message from remote failed, will be reconnect"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8] [error="ErrorType: MessageReceiveFailed, Reason: rpc error: code = Unknown desc = ErrorType: Unknown, Reason: Target 67873c92-0183-4c08-bfe3-77138d9532c8 not found"]
...
[2024/12/30 17:49:38.963 +08:00] [INFO] [controller.go:514] ["send coordinator bootstrap request"] [to=67873c92-0183-4c08-bfe3-77138d9532c8]
[2024/12/30 17:49:38.968 +08:00] [INFO] [reactor_state.go:77] ["remote capture offline"] [info="{\"id\":\"67873c92-0183-4c08-bfe3-77138d9532c8\",\"address\":\"10.2.6.80:8300\",\"version\":\"v8.2.0-master\",\"git-hash\":\"65f34ab1af943cf517863e89ae5dfb87d96ec009\",\"deploy-path\":\"/root/tidb-deploy/cdc-8300/bin/cdc\",\"start-timestamp\":1735551641}"] [role=capture-manager]
[2024/12/30 17:49:38.968 +08:00] [INFO] [module_node_manager.go:93] ["server change detected"]
[2024/12/30 17:49:38.968 +08:00] [INFO] [message_center.go:176] ["remove remote target from message center"] [local=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8]
[2024/12/30 17:49:38.969 +08:00] [INFO] [target.go:182] ["Close remote target"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8] [addr=10.2.6.80:8300]
[2024/12/30 17:49:39.463 +08:00] [INFO] [controller.go:514] ["send coordinator bootstrap request"] [to=67873c92-0183-4c08-bfe3-77138d9532c8]
[2024/12/30 17:49:40.946 +08:00] [INFO] [target.go:282] ["reconnect to remote target"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8]
[2024/12/30 17:49:40.946 +08:00] [INFO] [target.go:252] ["Cannot establish event grpc stream"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8] [error="rpc error: code = Canceled desc = context canceled"]
[2024/12/30 17:49:40.946 +08:00] [WARN] [target.go:202] ["received message from remote failed, will be reconnect"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8] [error="ErrorType: ConnectionFailed, Reason: Cannot open event grpc stream, error: rpc error: code = Canceled desc = context canceled"]
[2024/12/30 17:49:42.947 +08:00] [INFO] [target.go:282] ["reconnect to remote target"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8]
[2024/12/30 17:49:42.948 +08:00] [INFO] [target.go:252] ["Cannot establish event grpc stream"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8] [error="rpc error: code = Canceled desc = context canceled"]
[2024/12/30 17:49:42.948 +08:00] [WARN] [target.go:202] ["received message from remote failed, will be reconnect"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8] [error="ErrorType: ConnectionFailed, Reason: Cannot open event grpc stream, error: rpc error: code = Canceled desc = context canceled"]
[2024/12/30 17:49:44.948 +08:00] [INFO] [target.go:282] ["reconnect to remote target"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8]
[2024/12/30 17:49:44.949 +08:00] [INFO] [target.go:252] ["Cannot establish event grpc stream"] [messageCenterID=740e2769-063c-4170-889a-f5bbcd02a71f] [remote=67873c92-0183-4c08-bfe3-77138d9532c8] [error="rpc error: code = Canceled desc = context canceled"]
[2024/12/30 17:49:44.949 +08:00] [INFO] [coordinator.go:150] ["log coordinaotr detect node removed"] [nodeId=67873c92-0183-4c08-bfe3-77138d9532c8]
[2024/12/30 17:49:44.949 +08:00] [WARN] [etcd_worker.go:294] ["EtcdWorker reactor tick took too long"] [duration=5.980571678s] [role=capture-manager]
[2024/12/30 17:49:44.949 +08:00] [INFO] [controller.go:140] ["coordinator is too slow"] [type=1] [duration=5.485943287s]
[2024/12/30 17:49:44.949 +08:00] [INFO] [controller.go:197] ["node changed"] [new=0] [removed=1]
[2024/12/30 17:49:44.949 +08:00] [INFO] [bootstrap.go:98] ["remove node from bootstrap"] [changefeed=coordinator] [status=1] [id=67873c92-0183-4c08-bfe3-77138d9532c8]
[2024/12/30 17:49:44.949 +08:00] [INFO] [controller.go:203] ["bootstrap done after removed some nodes"]
[2024/12/30 17:49:44.949 +08:00] [INFO] [controller.go:230] ["all nodes have sent bootstrap response"] [size=1]
[2024/12/30 17:49:44.949 +08:00] [INFO] [controller.go:234] ["received bootstrap response"] [server=740e2769-063c-4170-889a-f5bbcd02a71f] [size=0]

After restart the server, the old server still alive for more than 20 seconds, this should be treated as unexpected.

Reproduce the cluster deployment and CDC restart sequence, then trace the lifecycle in reactor_state.go, message_center.go, target.go, controller.go, coordinator.go, and bootstrap.go. Start with the logged remote-capture and target-removal flow, and use the repeated reconnect and coordinator messages to identify where shutdown remains delayed. Done means the old server exits promptly after its replacement starts, without the reported 20-second survival.