apache / apache/pulsar

[Doc] Compaction improvements

Open
#19,335 3 comments 0 reactions 0 assignees View on GitHub
doc-required Stale
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### What issue do you find in Pulsar docs?

Compaction documents ( [concepts](https://pulsar.apache.org/docs/2.11.x/concepts-topic-compaction/) and [cookbook](https://pulsar.apache.org/docs/2.11.x/cookbooks-compaction/)) doesn't explains about [how to gather the amount of keys](https://github.com/apache/pulsar/blob/badd69b8e012162feb28c86ca9de527cbfd1ac22/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L2246-L2254) in use? neither [how long it took to compact](https://github.com/apache/pulsar/blob/82237d3684fe506bcb6426b3b23f413422e6e4fb/pulsar-broker/src/main/java/org/apache/pulsar/compaction/CompactionRecord.java#L70-L73) it or when it was [done](https://github.com/apache/pulsar/blob/82237d3684fe506bcb6426b3b23f413422e6e4fb/pulsar-broker/src/main/java/org/apache/pulsar/compaction/CompactionRecord.java#L75-L80)

Neither it explains how to delete a key to avoid the number of keys increases too much with tombstones as performed in the [tests.](https://github.com/apache/pulsar/blob/05e6f5e3c71426ee30a8ad1852456f7be897bdf3/pulsar-broker/src/test/java/org/apache/pulsar/compaction/CompactionTest.java#L684-L692)

### What is your suggestion?

Improve this on the documentation.

Even mention the side effects over a huge amount of topics.

### Any reference?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with the linked compaction concepts and cookbook pages, then read the referenced PersistentTopic.java, CompactionRecord.java, and CompactionTest.java sections. Update the documentation to explain gathering key counts, compaction duration and completion, deleting keys with tombstones, and side effects across many topics; done means these cases are covered in both relevant pages.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.