matrixorigin / matrixorigin/matrixone

[Bug]: query_client morpc backend missing read timeout causes goroutine leak on unresponsive CN

Open
#25,191 0 comments 0 reactions 1 assignee Claimed by @jiangxinmeng1 View on GitHub
deferred kind/bug
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

### Branch & Commit
- Branch: `main`
- Commit: `0a665318fb`

### Actual Behavior
多 CN 场景下,当远端 CN 假死(TCP 连接存活但不响应),`query_client` 创建的 morpc backend 的 `readLoop` goroutine 在 `conn.Read()` 上永久阻塞。

**Root Cause**:`getBackendOptions()` 未设置 `WithBackendReadTimeout`。仅 `MethodBasedClient` 路径(`method_based.go:294`)设置了 `internalTimeout=10s`,`query_client` 走 `Config.NewClient()` 路径时缺少该设置。

### Environments
- Multi-CN cluster >= 2 nodes
- Unresponsive CN (TCP alive, no data)

### Steps to Reproduce
1. 启动多 CN 集群
2. 模拟远端 CN 假死(例如 iptables drop 出方向包但保留 TCP 连接)
3. 对假死 CN 调用 `query_client.SendMessage()`
4. 观察 backend `readLoop` goroutine 行为

### Expected Behavior
Backend 应在 `readTimeout` 内检测到对端无响应并退出 `readLoop`,释放连接资源供 pool 回收或重建。

### Key Code Paths
```
query_client.go:83 NewQueryClient()
→ cfg.go:129 Config.NewClient()
→ cfg.go:195-206 getBackendOptions() ← 缺少 WithBackendReadTimeout
→ backend.go:660 readLoop: conn.Read(goetyy.ReadOptions{Timeout: 0}) ← 永久阻塞
```

**涉及文件**:
- `pkg/queryservice/client/query_client.go:83` — `NewQueryClient` 创建路径
- `pkg/common/morpc/cfg.go:195-206` — `getBackendOptions()` 缺少 read timeout
- `pkg/common/morpc/method_based.go:294` — 已有正确设置可供参考
- `pkg/common/morpc/backend.go:660` — `readLoop` 中使用 `readTimeout` 字段

### Suggested Fix
在 `getBackendOptions()` 中默认添加 `WithBackendReadTimeout`,或在 `query_client` 创建时显式设置,对齐 `MethodBasedClient` 的行为。

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.