apache / apache/pulsar

stats-internal throws 500 with message No such ledger exists on Metadata Server

Open
#14,533 5 comments 0 reactions 0 assignees View on GitHub
help wanted lifecycle/stale Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
I was getting MissingLogFileException for one of the topic when starting the consumer, when looked into the stats-internal of the topic I found the ledgers for which log file was missing belongs to schemaledger. I deleted those ledgers manually, post that the exception stopped coming for the topic but the stats-internal command started failing with 500 error code.
Below is the response I get when running the command.

No such ledger exists on Metadata Server
Reason: HTTP 500 Internal Server Error

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: [e.g. iOS]

**Additional context**
I was upgrading my cluster from 2.5.2 to 2.8.2 which failed because of the TLS protocol version error as I was using old version of Java8. During this I reverted the upgrade and since then I am getting the MissingLogFileException for most of the ledgers.
All those dependencies are now fixed and currently I am running my cluster on 2.8.2

Contributor guide

Open the contributing guide

Research direction

Start with the stats-internal command and the MissingLogFileException path described in the report, then trace the metadata-server lookup that returns “No such ledger exists.” Reproduce the HTTP 500 using the reported ledger state and verify that stats-internal handles missing ledgers as intended.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.