cockroachdb / cockroachdb/cockroach

kvserver: reduce reproposals of AddSSTable and other large commands

Open
#98,563 3 comments 0 reactions 0 assignees View on GitHub
C-performance T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

We've seen that Raft can run nodes out of memory during overload, in particular with large commands like `AddSSTable` (see e.g. #73376). One possible contributing factor here is reproposals, where an overloaded Raft group that takes more than 3 seconds to ack a proposal may continually stuff the same command into Raft over and over, worsening the overload. We should consider throttling reproposals of large commands.

Jira issue: CRDB-25334

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the overload and reproposal context in #73376, then follow the linked Jira issue CRDB-25334 for the intended scope. Investigate how large commands such as AddSSTable are reproposed during delayed acknowledgements, and define the throttling behavior and overload-memory evidence needed to consider the issue done.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.