cockroachdb / cockroachdb/cockroach
kvserver: add a roachtest repro for https://github.com/cockroachdb/cockroach/issues/160011
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
We should add a roachtest that repros what we saw in https://github.com/cockroachdb/cockroach/issues/160011. It should run 20k warehouses, tpcc inserts, 3 nodes.
- Claude said the closest roachtest we have is tpcc/published/headroom/25k) which runs tpcc, 20k warehouses, 3 nodes, 30 mins, 500 active workers.
- Find the relevant env files in https://cockroachlabs.slack.com/archives/C048HDZJSAY/p1768595332417589. The exact repro is:
```
git clone https://github.com/cockroachlabs/perf-efficiency-team
// use the given env files for cluster and workload configuration
./rpsak.sh create-only radu-tpcc
./rpsak.sh repeated radu-tpcc
```
- It should do be similar to below:
```
cd $crl
# Variables
CLUSTER=wenyicockroachlabscom-radurepro
NODES=3
WORKLOAD_NODE=4
ZONE=us-east1-d
MACHINE=n2-standard-32
SSDS=4
WAREHOUSES=20000
WORKERS=500
DURATION=30m
# 1. Create cluster
roachprod create $CLUSTER \
--nodes 4 \
--clouds gce \
--gce-machine-type $MACHINE \
--gce-zones $ZONE \
--gce-local-ssd-count $SSDS \
--username $(whoami) \
--lifetime 240h
# 2. Stage cockroach binary
roachprod stage $CLUSTER cockroach
# OR: roachprod put $CLUSTER ./cockroach-$SHA ./cockroach
# 3. Stage workload binary on workload node
roachprod stage $CLUSTER:$WORKLOAD_NODE workload
# 4. Start CRDB
roachprod start $CLUSTER:1-$NODES --secure
sleep 15
# 5. Create tpcc database
roachprod sql $CLUSTER:1 --secure -- -e "CREATE DATABASE IF NOT EXISTS tpcc;"
# 6. Get connection URL
PGURL=$(roachprod pgurl $CLUSTER:1 --secure | sed "s/^'//; s/'$//")
# 7. Initialize TPC-C with workload tool (CRDB kit)
roachprod ssh $CLUSTER:$WORKLOAD_NODE -- \
"./workload init tpcc --warehouses=$WAREHOUSES '$PGURL'"
# 8. Run TPC-C workload
roachprod ssh $CLUSTER:$WORKLOAD_NODE -- \
"./workload run tpcc \
--warehouses=$WAREHOUSES \
--workers=$WORKERS \
--wait=false \
--duration=$DURATION \
'$PGURL'"
# 9. Check for large ranges
roachprod ssh $CLUSTER:1-$NODES -- \
"grep -E 'initiating a split.*above threshold' logs/cockroach.log | tail -20"
# 10. Destroy
roachprod destroy $CLUSTER
```
Jira issue: CRDB-58876
Contributor guide
Assessment
This issue has not been assessed yet.