apache / apache/incubator-seata

rpcMergeMessageSend_TMROLE_1、rpcMergeMessageSend_RMROLE_1的cpu使用率过高

Open
#6,041 5 comments 0 reactions 0 assignees View on GitHub
good first issue task: help-wanted type: optimize
Dominant language
Java
Stars
26k
Forks
8.8k
Avg merge
1d 8h
Merged PRs (30d)
4

Description

- [ ] I have searched the [issues](https://github.com/seata/seata/issues) of this repository and believe that this is not a duplicate.

### Ⅰ. Issue Description
seata版本:1.4.2
服务配置:-Xms512m -Xmx1024m -XX:MaxNewSize=128m
因一直收到线上某服务cpu使用率过高的告警,就先nacos进行下线,准备重启大法,发现还是一直告警。然后arthas排查,最后发现无流量情况下,
rpcMergeMessageSend_TMROLE_1、rpcMergeMessageSend_RMROLE_1这2线程的cpu使用率过高
### Ⅱ. Describe what happened
"rpcMergeMessageSend_TMROLE_1" Id=31 cpuUsage=31% TIMED_WAITING on java.lang.Object@1c0c220d
at java.lang.Object.wait(Native Method)
- waiting on java.lang.Object@1c0c220d
at io.seata.core.rpc.netty.AbstractRpcRemotingClient$MergedSendRunnable.run(AbstractRpcRemotingClient.java:301)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)

Number of locked synchronizers = 1
- java.util.concurrent.ThreadPoolExecutor$Worker@4998e74b

"rpcMergeMessageSend_RMROLE_1" Id=34 cpuUsage=30% TIMED_WAITING on java.lang.Object@25b70c1e
at java.lang.Object.wait(Native Method)
- waiting on java.lang.Object@25b70c1e
at io.seata.core.rpc.netty.AbstractRpcRemotingClient$MergedSendRunnable.run(AbstractRpcRemotingClient.java:301)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)

Number of locked synchronizers = 1
- java.util.concurrent.ThreadPoolExecutor$Worker@2b08772d
-
If there is an exception, please attach the exception trace:

```
Just paste your stack trace here!
```

### Ⅲ. Describe what you expected to happen

### Ⅳ. How to reproduce it (as minimally and precisely as possible)

1. xxx
2. xxx
3. xxx

Minimal yet complete reproducer code (or URL to code):

### Ⅴ. Anything else we need to know?

### Ⅵ. Environment:

- JDK version(e.g. `java -version`):1.8
- Seata client/server version: 1.0.0/1.4.2
- Database version:
- OS(e.g. `uname -a`):
- Others:

Contributor guide

Open the contributing guide

Research direction

Start with io.seata.core.rpc.netty.AbstractRpcRemotingClient.java at the MergedSendRunnable.run method around line 301, then use Arthas with JDK 8 to inspect the two named threads while the service has no traffic. Done means identifying and correcting the idle CPU behavior and verifying that rpcMergeMessageSend_TMROLE_1 and rpcMergeMessageSend_RMROLE_1 no longer consume excessive CPU without traffic.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.