micropython / micropython/micropython-lib

UMQTT: Library stops receiving broker responses after a subscribed message arrives

Open
#951 9 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.9k
Forks
1.1k
Avg merge
7d 6h
Merged PRs (30d)
3

Description

MicroPython v1.24.1
Hardware: Raspberry Pi Pico W

Attempted using umqtt.simple v1.5.0 installed via Thonny package manager, as well as manually installing most recent simple.py and robust.py files from GitHub.


I have an IoT application I'm creating with a Raspberry Pi Pico W, and have been trying to get either the umqtt.simple or .robust tools to work for several days now, but am running into a wall where upon receipt of certain messages, the library just freezes and will no longer receive ANY messages until the client connection is completely torn-down and reconnected. I initially thought that this might be some kind of networking issue or socket issue, but it appears to be a function of inbound messages arriving on a subscribed topic.

I've modified simple.py to add a debug print() inside of wait_msg() (right under the "res = self.sock.read(1)") so I can see the raw byte(s) coming back from the socket read, and try and determine where the problem is coming in. I've also implemented a standard ICMP Ping to be able to test network connectivity at the same time as MQTT connectivity, to rule out network-related issues. It's resulted in useful debug output, but I'm no closer to a solution.

Simplified code example:

import network
import ubinascii
import time

from icmp import ping
from umqtt.simple import MQTTClient
from config import *

wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect(WIFI_SSID, WIFI_PASSWORD)
time.sleep(5)

def mqtt_callback(topic, msg, retained=False, properties=None):
  message = msg.decode()
  topic = topic.decode()
  print(f"Received message: {message} on topic: {topic}, Retained: {retained}, Properties: {properties}")

client_id = ubinascii.hexlify(machine.unique_id()).decode()
client = MQTTClient(client_id, MQTT_BROKER, MQTT_PORT, MQTT_USER, MQTT_PASSWORD)
client.set_callback(mqtt_callback)
client.connect(clean_session=False)
client.subscribe(MQTT_TOPIC_FAN_CONTROL)
client.subscribe(MQTT_DISCOVERY_REQUEST_TOPIC)
client.subscribe(MQTT_TOPIC_FURNACE_STATE)

ping_counter = 0
while True:
  if ping_counter >= 10:
    print("Pinging ICMP and MQTT")
    ping()
    client.ping()
    ping_counter = 0
  else:
    ping_counter += 1

  client.check_msg()
  time.sleep(1)

So it checks for a new message every second, and every 10 seconds it both ICMP pings the MQTT broker host, and MQTT pings the broker service.

Initial output:

Received byte from MQTT server: b'\x90'
Received byte from MQTT server: b'\x90'
Received byte from MQTT server: b'\x90'
Received byte from MQTT server: b'1'
Received message: ON on topic: testtopic/result, Retained: True, Properties: False
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
...
Pinging ICMP and MQTT
ICMP TX(18) : 08 00 16 f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46 
ICMP RX(18) : 00 00 1e f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46 
IP Version  = v4
IP Header   = 20
IP Length   = 44
Protocol    = 0x01
Source      = 192.168.1.237
Destination = 192.168.1.131
ICMP Length = 24
ICMP Type   = 00
ICMP Code   = 00
Checksum    = Passed
Identifier  = 1234
Sequence N  = 0001
ICMP Data   = 0123456789ABCDEF

Received byte from MQTT server: None
Received byte from MQTT server: b'\xd0'
Received byte from MQTT server: None
Received byte from MQTT server: None

So, setup proceeds as expected. I subscribe to 3 topics, and get back 3x "\x90" acknowledgement messages. I've configured the messages I'm sending to the broker to be retained, so it then immediately receives a "1" and hits the callback for a received message. A few seconds go by, and we hit our first Ping. The ICMP ping succeeds, so the network is up. A second later, the MQTT server replies with a "\xd0" ping acknowledgement byte. All good so far!

This will run for some arbitrary length of time just fine. And, as far as I can tell, if no messages are sent to subscribed topics, it will run forever. But eventually, after some arbitrary number of messages received, the library just stops responding, even though the network is verifiably up, as is the MQTT broker:

Pinging MQTT Server.
ICMP TX(18) : 08 00 16 f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46 
ICMP RX(18) : 00 00 1e f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46 
IP Version  = v4
IP Header   = 20
IP Length   = 44
Protocol    = 0x01
Source      = 192.168.1.237
Destination = 192.168.1.131
ICMP Length = 24
ICMP Type   = 00
ICMP Code   = 00
Checksum    = Passed
Identifier  = 1234
Sequence N  = 0001
ICMP Data   = 0123456789ABCDEF
Received byte from MQTT server: None
Received byte from MQTT server: b'\xd0'
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None   **<==== MESSAGE RECEIVED APPROXIMATELY HERE**
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Pinging MQTT Server.
ICMP TX(18) : 08 00 16 f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46 
ICMP RX(18) : 00 00 1e f4 12 34 00 01 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46 
IP Version  = v4
IP Header   = 20
IP Length   = 44
Protocol    = 0x01
Source      = 192.168.1.237
Destination = 192.168.1.131
ICMP Length = 24
ICMP Type   = 00
ICMP Code   = 00
Checksum    = Passed
Identifier  = 1234
Sequence N  = 0001
ICMP Data   = 0123456789ABCDEF
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Pinging MQTT Server.
ICMP TX(18) : 08 00 16 f3 12 34 00 02 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46 
ICMP RX(18) : 00 00 1e f3 12 34 00 02 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46 
IP Version  = v4
IP Header   = 20
IP Length   = 44
Protocol    = 0x01
Source      = 192.168.1.237
Destination = 192.168.1.131
ICMP Length = 24
ICMP Type   = 00
ICMP Code   = 00
Checksum    = Passed
Identifier  = 1234
Sequence N  = 0002
ICMP Data   = 0123456789ABCDEF
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None
Received byte from MQTT server: None


The above output snip shows the point where a message is received. We have a good ping at the beginning. Approximately 5 seconds later, a message is sent on a subscribed topic. The message never arrives, but more importantly neither does anything else, ever again. Even though pings are still being sent, the acknowledgement from the server is never received. ICMP pings to the MQTT broker continue to succeed. The only way to restore operation is to tear down and reconnect the entire MQTT client.

I'm completely baffled and have been unable to determine the cause. Any assistance would be appreciated.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the umqtt.simple and umqtt.robust simple.py and robust.py files, especially wait_msg() and the socket reads shown in the report. Reproduce on MicroPython v1.24.1 with a Raspberry Pi Pico W after a subscribed message arrives, then verify that subsequent MQTT messages and ping acknowledgements continue to be received without reconnecting.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.