org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId

Hi,

I'm getting error logs in pulsar bookkeeper and because of this I'm not getting acknowledged from the pulsar whether the job is pushed successfully or not. Since there is no acknowledgment the jobs are lost once the instance is restarted. Following is the error trace,

Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.013 [pulsar-io-23-13] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37758 lastEntry:37758
Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.439 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37760 lastEntry:37760
Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.540 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37761 lastEntry:37761
Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.839 [pulsar-io-23-2] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.014 [pulsar-io-23-13] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.214 [pulsar-io-23-4] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42118 firstEntry:866 lastEntry:866
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.240 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.414 [pulsar-io-23-13] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.440 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762

I have checked the same exception in the bookkeeper code,
From this it is understood that the lastEntry is greater than the lastAddConfirmed. We aren't sure why this is happening, is there a way we can over come it ?

Start with the org.apache.bookkeeper.client.LedgerHandle ReadException path and the BookKeeper code referenced in the report. Compare the logged ledgerId, firstEntry, lastEntry, and lastAddConfirmed values, then gather the missing broker, ledger, and reproduction details. Done means identifying the cause and documenting or validating a concrete resolution for the lost acknowledgments.

Written by the indexing model from the issue text.