eclipse-paho / eclipse-paho/paho.mqtt.java

Dead connection stuck forever if connection hangs before MqttConnack

Open
#1,028 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
2.3k
Forks
919
PR merge metrics
No merged PRs in 30d

Description

Please fill out the form below before submitting, thank you!

- [ X] Bug exists Release Version 1.2.5 ( Master Branch)
- [ ] Bug exists in MQTTv3 Client on Snapshot Version 1.2.6-SNAPSHOT (Develop Branch) - Probably
- [ ] Bug exists in MQTTv5 Client on Snapshot Version 1.2.6-SNAPSHOT (Develop Branch) - Probably

Diagnostics:
-- Mqtt Rec/Mqtt Snd/Mqtt Rec threads are running
-- Mqtt Ping sender thread has not been started yet
-- The connection is dead and has been hanging for days
-- Mqtt Rec thread keeps looping into readMqttWireMessage

My Analysis:
-- The keep alive mechanism fails and a dead connection appears to hang forever if the socket hangs after the connection is established but before MqttConnack is received. The keep-alive The pingSender keep-alive mechanism is initialized only upon a successful receipt of MqttConnack message. If the client connects but then doesn't receive a response to MqttConnect, there does not appear to be a time-out disconnect mechanism.

Partial Thread Dump:
```
"MQTT Rec: ...." daemon prio=5 tid=22 Native
| group="main" sCount=1 dsCount=0 flags=1 obj=0x12d022a0 self=0x97a5a000
| sysTid=12207 nice=0 cgrp=default sched=0/0 handle=0x96b7f970
| state=S schedstat=( 1688215063136 59566797290 2969618 ) utm=141496 stm=27325 core=3 HZ=100
| stack=0x96a7c000-0x96a7e000 stackSize=1042KB
| held mutexes=
native: #00 pc 00053d18 /system/lib/libc.so (__ppoll+20)
native: #01 pc 000234ed /system/lib/libc.so (poll+48)
native: #02 pc 0002122f /system/lib/libjavacrypto.so (sslSelect(_JNIEnv*, int, _jobject*, conscrypt::AppData*, int)+166)
native: #03 pc 00021589 /system/lib/libjavacrypto.so (sslRead(_JNIEnv*, ssl_st*, _jobject*, _jobject*, char*, int, conscrypt::SslError*, int)+572)
native: #04 pc 00019efb /system/lib/libjavacrypto.so (NativeCrypto_SSL_read(_JNIEnv*, _jclass*, long long, _jobject*, _jobject*, _jobject*, _jbyteArray*, int, int, int)+346)
at com.android.org.conscrypt.NativeCrypto.SSL_read(Native method)
at com.android.org.conscrypt.NativeSsl.read(NativeSsl.java:399)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket$SSLInputStream.read(ConscryptFileDescriptorSocket.java:546)
- locked <0x0dad74e4> (a java.lang.Object)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket$SSLInputStream.read(ConscryptFileDescriptorSocket.java:517)
at java.io.DataInputStream.readByte(DataInputStream.java:268)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttInputStream.readMqttWireMessage(MqttInputStream.java:92)
at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:137)
at java.lang.Thread.run(Thread.java:764)

"MQTT Snd: .... daemon prio=5 tid=23 Waiting
| group="main" sCount=1 dsCount=0 flags=1 obj=0x12d02c98 self=0x97a5ac00
| sysTid=12208 nice=0 cgrp=default sched=0/0 handle=0x96973970
| state=S schedstat=( 1648002 341229 6 ) utm=0 stm=0 core=2 HZ=100
| stack=0x96870000-0x96872000 stackSize=1042KB
| held mutexes=
at java.lang.Object.wait(Native method)
- waiting on <0x08d06d4d> (a java.lang.Object)
at org.eclipse.paho.client.mqttv3.internal.ClientState.get(ClientState.java:833)
- locked <0x08d06d4d> (a java.lang.Object)
at org.eclipse.paho.client.mqttv3.internal.CommsSender.run(CommsSender.java:129)
at java.lang.Thread.run(Thread.java:764)

"MQTT Call: ... Waiting
| group="main" sCount=1 dsCount=0 flags=1 obj=0x12d03388 self=0x97a5b200
| sysTid=12209 nice=0 cgrp=default sched=0/0 handle=0x9686d970
| state=S schedstat=( 453154 0 1 ) utm=0 stm=0 core=0 HZ=100
| stack=0x9676a000-0x9676c000 stackSize=1042KB
| held mutexes=
at java.lang.Object.wait(Native method)
- waiting on <0x089a3002> (a java.lang.Object)
at org.eclipse.paho.client.mqttv3.internal.CommsCallback.run(CommsCallback.java:181)
- locked <0x089a3002> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:764)
```

Contributor guide

Open the contributing guide

Research direction

Start with org.eclipse.paho.client.mqttv3.internal.wire.MqttInputStream.readMqttWireMessage and CommsReceiver.run, then trace when the ping sender is initialized after MqttConnack. Reproduce the pre-CONNACK hang described by the thread dump; done means a connection that receives no CONNACK is disconnected rather than remaining blocked indefinitely.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.