apache / apache/incubator-seata
rpcMergeMessageSend_TMROLE_1、rpcMergeMessageSend_RMROLE_1的cpu使用率过高
- Dominant language
- Java
- Stars
- 26k
- Forks
- 8.8k
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 4
Description
- [ ] I have searched the [issues](https://github.com/seata/seata/issues) of this repository and believe that this is not a duplicate.
### Ⅰ. Issue Description
seata版本:1.4.2
服务配置:-Xms512m -Xmx1024m -XX:MaxNewSize=128m
因一直收到线上某服务cpu使用率过高的告警,就先nacos进行下线,准备重启大法,发现还是一直告警。然后arthas排查,最后发现无流量情况下,
rpcMergeMessageSend_TMROLE_1、rpcMergeMessageSend_RMROLE_1这2线程的cpu使用率过高
### Ⅱ. Describe what happened
"rpcMergeMessageSend_TMROLE_1" Id=31 cpuUsage=31% TIMED_WAITING on java.lang.Object@1c0c220d
at java.lang.Object.wait(Native Method)
- waiting on java.lang.Object@1c0c220d
at io.seata.core.rpc.netty.AbstractRpcRemotingClient$MergedSendRunnable.run(AbstractRpcRemotingClient.java:301)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Number of locked synchronizers = 1
- java.util.concurrent.ThreadPoolExecutor$Worker@4998e74b
"rpcMergeMessageSend_RMROLE_1" Id=34 cpuUsage=30% TIMED_WAITING on java.lang.Object@25b70c1e
at java.lang.Object.wait(Native Method)
- waiting on java.lang.Object@25b70c1e
at io.seata.core.rpc.netty.AbstractRpcRemotingClient$MergedSendRunnable.run(AbstractRpcRemotingClient.java:301)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Number of locked synchronizers = 1
- java.util.concurrent.ThreadPoolExecutor$Worker@2b08772d
-
If there is an exception, please attach the exception trace:
```
Just paste your stack trace here!
```
### Ⅲ. Describe what you expected to happen
### Ⅳ. How to reproduce it (as minimally and precisely as possible)
1. xxx
2. xxx
3. xxx
Minimal yet complete reproducer code (or URL to code):
### Ⅴ. Anything else we need to know?
### Ⅵ. Environment:
- JDK version(e.g. `java -version`):1.8
- Seata client/server version: 1.0.0/1.4.2
- Database version:
- OS(e.g. `uname -a`):
- Others:
Contributor guide
Research direction
Start with io.seata.core.rpc.netty.AbstractRpcRemotingClient.java at the MergedSendRunnable.run method around line 301, then use Arthas with JDK 8 to inspect the two named threads while the service has no traffic. Done means identifying and correcting the idle CPU behavior and verifying that rpcMergeMessageSend_TMROLE_1 and rpcMergeMessageSend_RMROLE_1 no longer consume excessive CPU without traffic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100