apache / apache/rocketmq

[Bug] TOCTOU race condition in MQClientInstance.brokerVersionTable access

Open
#10,214 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
22.6k
Forks
12k
Avg merge
2d 20h
Merged PRs (30d)
26

Description

### Before Creating the Bug Report

- [x] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions).

- [x] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate.

- [x] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.

### Runtime platform environment

All platforms

### RocketMQ version

develop branch (latest)

### JDK Version

JDK 8+

### Describe the Bug

In `MQClientInstance.java`, the `brokerVersionTable` (a `ConcurrentHashMap`) is accessed using a non-atomic check-then-act pattern in multiple locations:

**Location 1 - `sendHeartbeatToBroker()` (line ~657):**
```java
if (!this.brokerVersionTable.containsKey(brokerName)) {
this.brokerVersionTable.put(brokerName, new ConcurrentHashMap<>(4));
}
this.brokerVersionTable.get(brokerName).put(addr, version);
```

**Location 2 - `sendHeartbeatToAllBrokerV2()` (line ~737):**
Same pattern as Location 1.

**Location 3 - `findBrokerVersion()` (line ~1304):**
```java
if (this.brokerVersionTable.containsKey(brokerName)) {
if (this.brokerVersionTable.get(brokerName).containsKey(brokerAddr)) {
return this.brokerVersionTable.get(brokerName).get(brokerAddr);
}
}
```

Between `containsKey()` and `get()`, another thread could remove the entry, causing a NullPointerException when the result of `get()` is dereferenced.

Notably, a similar operation in the same class at line ~814 already uses the correct pattern:
```java
ConcurrentHashMap inner = MQClientInstance.this.brokerVersionTable
.computeIfAbsent(brokerName, k -> new ConcurrentHashMap<>(4));
inner.put(brokerAddr, version);
```

### Steps to Reproduce

The race condition occurs under concurrent heartbeat sending and broker disconnection. When a broker is removed from the version table concurrently with a heartbeat send or version lookup, a NullPointerException can occur.

### What Did You Expect to See?

Thread-safe access to `brokerVersionTable` using atomic operations like `computeIfAbsent` and local variable caching.

### What Did You See Instead?

Non-atomic check-then-act pattern that can result in NullPointerException under concurrent access.

### Additional Context

Related to previously reported #8743 (closed by stale bot without fix).

Contributor guide

Open the contributing guide

Research direction

Start in MQClientInstance.java at sendHeartbeatToBroker(), sendHeartbeatToAllBrokerV2(), and findBrokerVersion(), then compare their brokerVersionTable access with the existing computeIfAbsent usage around line 814. The work is done when these paths no longer use unsafe check-then-act access and concurrent heartbeat sending, broker removal, and version lookup cannot produce the described NullPointerException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.