apache / apache/pulsar

bookkeeper storage ledgers did not auto delete [Bug]

Open
#20,320 1 comment 1 reaction 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Version

System: Centos7.6
Pulsar: 2.8.2

### Minimal reproduce step

All the namespaces/topics we set retention policy and ttl.
During pulsar manager we can see all namespace take about 25G storage size:
![image](https://github.com/apache/pulsar/assets/23557780/75724dcc-45d5-4d05-a4d0-7eac69fafc27)

But in system it used nearly 100G
![image](https://github.com/apache/pulsar/assets/23557780/a4e88aa3-f8fb-4b55-aaf7-80e291a45ed3)

I don't know why, i already use "curl -XPUT 127.0.0.1:8888/api/v1/bookie/gc" try release disk space and it not work.

### What did you expect to see?

Disk usage similar with what i see in pulsar manage.

### What did you see instead?

about 90G instead of 25G

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the storage discrepancy with namespace retention and TTL enabled, then investigate the bookie GC entry point at /api/v1/bookie/gc and compare its result with the Pulsar Manager usage. Done means expired ledger data is removed and system disk usage is consistent with the reported namespace storage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.