oxidecomputer / oxidecomputer/omicron
Reading an inventory collection from the database does not paginate correctly
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
Inventory collections are stored in a few tables in the database, keyed by the collection ID, the sled config ID, and a few other things depending on the table. When we read the collection back out, we read all those different tables in pages, but we're currently paginating only on a suffix of their primary keys. That means it's technically possible to miss items which happen right on a page boundary and have different values for the part of the PK we're not using in the keyset pagination.
We don't observe this today because you'd need to have more items in a table than the SQL_BATCH_SIZE (1000) we use today for pagination. I've written a regression test that uses a tiny page size to prove we do miss items.
We should migrate these to using paginated_multicolumn or similar, to ensure we paginate across the whole PK.
Originally posted by @jgallagher in https://github.com/oxidecomputer/omicron/pull/11114#discussion_r3874607317
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the inventory collection database read path and the regression test described in the issue. Inspect how each table currently uses suffix-only keyset pagination, then verify the full primary key is covered using paginated_multicolumn or a similar approach; done means the small-page regression test no longer misses items at page boundaries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100