quickwit-oss / quickwit-oss/quickwit
Migration number 25 causes CPU spike and lock contention on splits tables during metastore startup
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Summary
Migration number 25 (add-split-size), introduced in #6035, adds a GENERATED ALWAYS AS ... STORED column to the splits table.
In our environment (~1.5M rows in splits table), we observed:
- CPU spike on the PostgreSQL instance
- Lock-related metrics spike
- Connection acquisition timeouts
Adding a GENERATED ALWAYS AS ... STORED column requires an exclusive lock to compute and store values for all existing rows, causing incoming queries to queue up.
Environment
- PostgreSQL metastore with ~1.5M rows in
splitstable - Upgrading to a version containing migration number 25 (
add-split-size)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with migration number 25 (add-split-size), introduced in #6035, and inspect how it adds the generated stored column to PostgreSQL’s splits table. Reproduce or measure the upgrade against a large table; done means the migration no longer causes the reported CPU, lock, and connection-acquisition spikes during metastore startup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100