[DISK Dos Attack Warning] should not use message publish time in checkMessageExpiry and AppendBrokerTimesampMetadataInterceptor should be config to brokerEntryMetadataInterceptors
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
all pulsar version
### Minimal reproduce step
1、brokerEntryMetadataInterceptors config not contain AppendBrokerTimesampMetadataInterceptor
2、create a topic and set retention and ttl to 5 min
3、pulsar client server change system time to some years later
4、publish message and start a pulsar consumer
5、stop pulsar consumer and continue publish message
6、message can not expire when ledger close and time reach
### What did you expect to see?
NA
### What did you see instead?
PersistentMessageExpiryMonitor
expireMessages(int messageTTLInSeconds) -->call Commands/MessageImpl.getEntryTimestamp(entry.getDataBuffer());
```
// get broker timestamp first if BrokerEntryMetadata is enabled with AppendBrokerTimestampMetadataInterceptor
BrokerEntryMetadata brokerEntryMetadata =Commands.parseBrokerEntryMetadataIfExist(headersAndPayloadWithBrokerEntryMetadata);
if (brokerEntryMetadata != null && brokerEntryMetadata.hasBrokerTimestamp()) {
return brokerEntryMetadata.getBrokerTimestamp();
}
// otherwise get the publish_time
return Commands.parseMessageMetadata(headersAndPayloadWithBrokerEntryMetadata).getPublishTime();
```
when brokerEntryMetadata.hasBrokerTimestamp() == false , pulsar server will use mesage publishTime
this will introduce vulnerability attack , server can not trust any client message data
Suggestions:
add AppendBrokerTimesampMetadataInterceptor to brokerEntryMetadataInterceptors
### Anything else?
Others:
in transaction scenario , even when we config brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor, we can not get offset , i am not sure if brokerEntryMetadataInterceptors works in transaction produce
if brokerEntryMetadataInterceptors not work
maybe we need verify client publish time when server receive mesage .
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Start with PersistentMessageExpiryMonitor and Commands/MessageImpl.getEntryTimestamp, then inspect brokerEntryMetadataInterceptors and AppendBrokerTimesampMetadataInterceptor. Reproduce the five-minute retention and TTL scenario described in the issue, including transaction production if relevant. Done means the expiry path and interceptor configuration handle messages without broker timestamps safely, with behavior verified by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100