killme2008 / killme2008/Metamorphosis

MetaQ接收数据客户端进入死循环

Open
#129 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.3k
Forks
672
PR merge metrics
No merged PRs in 30d

Description

情况偶现,症状就是程序不退出,进入死循环,CPU使用率100%,关键的堆栈如下:

"Thread-48" daemon prio=10 tid=0x00007f47a8048000 nid=0x73d2 in Object.wait() [0x00007f4796dec000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1289)
- locked <0x000000077858ab88> (a java.lang.Thread)
at com.taobao.metamorphosis.client.consumer.SimpleFetchManager.interruptRunners(SimpleFetchManager.java:146)
at com.taobao.metamorphosis.client.consumer.SimpleFetchManager.stopFetchRunner(SimpleFetchManager.java:128)
at com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer.shutdown(SimpleMessageConsumer.java:162)
- locked <0x0000000776bdde10> (a com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer)
at com.taobao.metamorphosis.client.MetaMessageSessionFactory.shutdown(MetaMessageSessionFactory.java:336)
- locked <0x0000000776bdde58> (a com.taobao.metamorphosis.client.MetaMessageSessionFactory)
at com.taobao.metamorphosis.client.MetaMessageSessionFactory$2.run(MetaMessageSessionFactory.java:244)

"fetch-Runner-0" daemon prio=10 tid=0x00007f4898002800 nid=0x6bdd runnable [0x00007f49294c5000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
- locked <0x000000073760cf20> (a java.lang.InterruptedException)
at java.lang.Throwable.(Throwable.java:250)
at java.lang.Exception.(Exception.java:54)
at java.lang.InterruptedException.(InterruptedException.java:57)
at java.lang.Object.wait(Native Method)
at com.taobao.gecko.core.nio.impl.SelectorManager.awaitReady(SelectorManager.java:147)
- locked <0x00000007702ecd40> (a com.taobao.gecko.core.nio.impl.SelectorManager)
at com.taobao.gecko.core.nio.impl.SelectorManager.registerChannel(SelectorManager.java:122)
at com.taobao.gecko.core.extension.GeckoTCPConnectorController.connect(GeckoTCPConnectorController.java:75)
at com.taobao.gecko.service.impl.DefaultRemotingClient.connect(DefaultRemotingClient.java:120)
at com.taobao.gecko.service.impl.DefaultRemotingClient.connect(DefaultRemotingClient.java:148)
- locked <0x0000000776b9dbc8> (a com.taobao.gecko.service.impl.DefaultRemotingClient)
at com.taobao.metamorphosis.client.RemotingClientWrapper.connectWithRef(RemotingClientWrapper.java:156)
- locked <0x0000000770384cb8> (a java.util.HashSet)
at com.taobao.metamorphosis.client.RemotingClientWrapper.connectWithRef(RemotingClientWrapper.java:192)
at com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer.fetch(SimpleMessageConsumer.java:330)
at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.processRequest(SimpleFetchManager.java:227)
at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.run(SimpleFetchManager.java:215)
at java.lang.Thread.run(Thread.java:745)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading SimpleFetchManager.java, SimpleMessageConsumer.java, and RemotingClientWrapper.java at the stack-trace locations, then trace the shutdown path alongside FetchRequestRunner.processRequest. Reproduce the intermittent shutdown failure if possible and use thread dumps to verify that shutdown completes without the fetch runner remaining blocked or consuming 100% CPU.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.