influxdata / influxdata/influxdb
Skip `WalPeriod`s that are empty
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
### Problem
https://github.com/influxdata/influxdb/pull/26223 resolved an issue that was leading to empty `WriteBatch`s being created, which therefore had min and max timestamps of `i64::MAX` and `i64::MIN`, respectively, and could produce corrupted WAL files (see https://github.com/influxdata/influxdb/issues/25650).
There may still be corrupted WAL files created on users' object stores, however, so we need to handle those without panic'ing and preventing server start where completely clearing the WAL from object store is not an option.
### Proposed solution
Add a check here: https://github.com/influxdata/influxdb/blob/2e1f7ad9cb8e6f3f972e93ad45353393700ddc2e/influxdb3_wal/src/object_store.rs#L160-L169
So that if the min and max timestamps are equal to `i64::MAX` and `i64::MIN`, respectively, we skip adding the `WalPeriod` to the buffer (but still handle snapshot if necessary). This will avoid the panic in https://github.com/influxdata/influxdb/issues/25650, but we also need to verify that skipping `WalPeriod`s does not lead to other downstream issues.
This might require some manual testing to get right since we resolved the bug that allowed for the generation of these corrupt WAL files in the first place.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in influxdb3_wal/src/object_store.rs around lines 160-169 and inspect how empty WAL periods and snapshots are handled. Reproduce or manually test with a corrupted WAL containing the specified extreme timestamps, then verify that server startup avoids the panic and that skipping the period causes no downstream issues.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100