apache / apache/pulsar

[DISK Dos Attack Warning] should not use message publish time in checkMessageExpiry and AppendBrokerTimesampMetadataInterceptor should be config to brokerEntryMetadataInterceptors

Open
#21,347 0 comments 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Version

all pulsar version

### Minimal reproduce step

1、brokerEntryMetadataInterceptors config not contain AppendBrokerTimesampMetadataInterceptor
2、create a topic and set retention and ttl to 5 min
3、pulsar client server change system time to some years later
4、publish message and start a pulsar consumer
5、stop pulsar consumer and continue publish message
6、message can not expire when ledger close and time reach

### What did you expect to see?

NA

### What did you see instead?

PersistentMessageExpiryMonitor
expireMessages(int messageTTLInSeconds) -->call Commands/MessageImpl.getEntryTimestamp(entry.getDataBuffer());

```
// get broker timestamp first if BrokerEntryMetadata is enabled with AppendBrokerTimestampMetadataInterceptor
BrokerEntryMetadata brokerEntryMetadata =Commands.parseBrokerEntryMetadataIfExist(headersAndPayloadWithBrokerEntryMetadata);
if (brokerEntryMetadata != null && brokerEntryMetadata.hasBrokerTimestamp()) {
return brokerEntryMetadata.getBrokerTimestamp();
}
// otherwise get the publish_time
return Commands.parseMessageMetadata(headersAndPayloadWithBrokerEntryMetadata).getPublishTime();
```

when brokerEntryMetadata.hasBrokerTimestamp() == false , pulsar server will use mesage publishTime
this will introduce vulnerability attack , server can not trust any client message data

Suggestions:
add AppendBrokerTimesampMetadataInterceptor to brokerEntryMetadataInterceptors

### Anything else?

Others:
in transaction scenario , even when we config brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor, we can not get offset , i am not sure if brokerEntryMetadataInterceptors works in transaction produce

if brokerEntryMetadataInterceptors not work
maybe we need verify client publish time when server receive mesage .

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with PersistentMessageExpiryMonitor and Commands/MessageImpl.getEntryTimestamp, then inspect brokerEntryMetadataInterceptors and AppendBrokerTimesampMetadataInterceptor. Reproduce the five-minute retention and TTL scenario described in the issue, including transaction production if relevant. Done means the expiry path and interceptor configuration handle messages without broker timestamps safely, with behavior verified by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.