oxidecomputer / oxidecomputer/omicron

cockroachdb can sometimes believe ranges are over-replicated based on a default, hard-coded config

Open
#10,786 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

In some cases, particularly around cold startup, CockroachDB will use a default, hard-coded config for ranges. This config specifies an RF of 3, and by using it Cockroach will think any affected ranges are over-replicated.

This is closely related to #10658, but is a different way through which a cold startup can result in temporarily over-replicated ranges. (It was found through the same reproducer script as that issue.)

This was found and fixed upstream in CockroachDB 23.1. See https://github.com/cockroachdb/cockroach/issues/98421 and https://github.com/cockroachdb/cockroach/pull/98422. Worth noting that 23.1 flipped over to the Apache 2.0 license as of May 16, 2026, so in principle we can pull the fix in without licensing issues. But it seems like doing this would require pulling in a bunch of pre-requisites as well, so we may want to do a more targeted fix instead.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the hard-coded default in pkg/config/zonepb/zone.go and compare the upstream CockroachDB issue 98421 and pull request 98422. Use the linked omicron-10658-repro reproducer to observe the cold-start behavior, then identify the required prerequisites or a targeted approach. Done means affected ranges no longer appear temporarily over-replicated under the reproducer.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.