eclipse-paho / eclipse-paho/paho.mqtt.python

MQTTMatcher: # wildcard does not match parent level (spec 4.7.1.2) - sport/# fails to match topic sport

Open Beginner friendly
#950 1 comment 0 reactions 0 assignees View on GitHub
Status: Available
Dominant language
Python
Stars
2.4k
Forks
742
Avg merge
12d 48m
Merged PRs (30d)
1

Description

## Summary

`MQTTMatcher.iter_match()` never applies the multi-level wildcard `#` at the level where `#` is the last segment of the filter, so filters such as `sport/#` fail to match the parent topic `sport`. The MQTT specification requires this match (MQTT 3.1.1 §4.7.1.2 / MQTT 5.0 §4.7.1.2: *"'sport/#' also matches the singular 'sport', since # includes the parent level"*).

This affects both the public helper `client.topic_matches_sub()` and filtered callback dispatch (`Client.message_callback_add`) in `paho.mqtt.client`.

*Note: this is a static analysis finding based on reading the source; no runtime execution was performed by the reporter.*

## Location

- File: `src/paho/mqtt/matcher.py`
- Function/Class: `MQTTMatcher.iter_match` (and its inner `rec` closure)
- Consumers: `topic_matches_sub()` and `_handle_on_message()` in `src/paho/mqtt/client.py`

## Problem

The trie built by `__setitem__` stores each filter segment as a child node, so the filter `"sport/#"` produces the chain `root → "sport" → "#"`, with the callback/value stored on the `"#"` node's `_content`.

Matching walks the topic `"sport"` as follows:

```python
def rec(node, i=0):
if i == len(lst):
if node._content is not None:
yield node._content
else:
part = lst[i]
if part in node._children:
for content in rec(node._children[part], i + 1):
yield content
if '+' in node._children and (normal or i > 0):
...
if '#' in node._children and (normal or i > 0):
content = node._children['#']._content
if content is not None:
yield content
```

The `'#'` check lives only in the `else` branch, i.e. while there are still topic levels left to consume, and it inspects the children of the node reached **so far**. When `rec` reaches the node corresponding to the final topic level (`i == len(lst)`), it yields only that node's own `_content` and returns without ever checking whether that node has a `"#"` child whose content should be produced.

Concretely, for filter `sport/#` and topic `sport`:

1. `rec(root, 0)`: consumes `"sport"` exactly, recurses into the `sport` node.
2. `rec(sport_node, 1)`: `i == len(lst)`, yields `sport_node._content` which is `None`.
3. Control returns; no other branch fires because the `'#'` child hangs off the `sport` node, not the root.

Result: nothing is yielded even though the subscription exists and brokers will deliver such messages.

## Trigger / Reproduction

Any of these static-demonstrable conditions:

```python
from paho.mqtt.client import topic_matches_sub

# Per MQTT 3.1.1/5.0 §4.7.1.2 these must all be True,
# but the current implementation returns False:
topic_matches_sub("sport/#", "sport") # False (bug)
topic_matches_sub("/#", "/") # False (bug)
```

Runtime consequence via filtered callbacks:

1. `client.message_callback_add("sport/#", cb)` then `client.subscribe("sport/#")`
2. A publisher sends to topic `sport` (no second level).
3. The broker forwards the message (server-side matching is correct), but `_handle_on_message()` calls `self._on_message_filtered.iter_match("sport")`, which yields no callback. Dispatch falls back to the generic `on_message` handler, or drops the message entirely if only the filtered callback was set.

## Expected Behavior

Per MQTT 3.1.1 §4.7.1.2 and MQTT 5.0 §4.7.1.2, `#` includes the parent level: `sport/#` must match both `sport` and every topic below `sport`.

## Actual Behavior

`iter_match()` returns an empty iterator for the parent-level case, so `topic_matches_sub("sport/#", "sport")` returns `False` and filtered callbacks registered for `sport/#` are not invoked when messages arrive on `sport`.

## Impact

- `message_callback_add()` users silently miss broker-delivered messages on parent topics; the generic fallback makes this hard to notice.
- Any user code using `topic_matches_sub()` for subscription logic (ACL checks, dedup, bridging) rejects spec-valid matches.
- Note the existing tests in `tests/test_matcher.py` do not cover the parent-level case in either direction, so this regression surface is untested today.

## Suggested Direction

In `iter_match`'s terminal branch (`i == len(lst)`), after yielding `node._content`, additionally check whether `node._children` contains a `"#"` node with non-`None` content and yield it (respecting the existing `$`-topic guard already expressed by `(normal or i > 0)` — for `$`-prefixed topics the first level must still not match wildcards).

## Evidence

- `src/paho/mqtt/matcher.py`: `MQTTMatcher.__setitem__` builds one trie level per `/`-separated segment; `iter_match.rec` only examines `node._children['#']` inside the non-terminal branch.
- `src/paho/mqtt/client.py`: `topic_matches_sub()` treats any `StopIteration` from `iter_match` as "no match"; `_handle_on_message()` dispatches filtered callbacks solely based on `iter_match` output.
- Spec: MQTT 3.1.1, section 4.7.1.2 "Multi-level wildcard": *"For example, if you subscribe to 'sport/#' … the subscription also matches 'sport'"* (identical wording in MQTT 5.0).

Contributor guide

Open the contributing guide

Research direction

Start with src/paho/mqtt/matcher.py, especially MQTTMatcher.iter_match and its rec closure, then review tests/test_matcher.py. Verify the parent-level cases such as sport/# matching sport and /# matching /, and confirm the existing topic_matches_sub and filtered callback behavior remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.