influxdata / influxdata/influxdb

Skip `WalPeriod`s that are empty

Open
#26,237 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

v3
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

### Problem

https://github.com/influxdata/influxdb/pull/26223 resolved an issue that was leading to empty `WriteBatch`s being created, which therefore had min and max timestamps of `i64::MAX` and `i64::MIN`, respectively, and could produce corrupted WAL files (see https://github.com/influxdata/influxdb/issues/25650).

There may still be corrupted WAL files created on users' object stores, however, so we need to handle those without panic'ing and preventing server start where completely clearing the WAL from object store is not an option.

### Proposed solution

Add a check here: https://github.com/influxdata/influxdb/blob/2e1f7ad9cb8e6f3f972e93ad45353393700ddc2e/influxdb3_wal/src/object_store.rs#L160-L169

So that if the min and max timestamps are equal to `i64::MAX` and `i64::MIN`, respectively, we skip adding the `WalPeriod` to the buffer (but still handle snapshot if necessary). This will avoid the panic in https://github.com/influxdata/influxdb/issues/25650, but we also need to verify that skipping `WalPeriod`s does not lead to other downstream issues.

This might require some manual testing to get right since we resolved the bug that allowed for the generation of these corrupt WAL files in the first place.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in influxdb3_wal/src/object_store.rs around lines 160-169 and inspect how empty WAL periods and snapshots are handled. Reproduce or manually test with a corrupted WAL containing the specified extreme timestamps, then verify that server startup avoids the panic and that skipping the period causes no downstream issues.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.