cockroachdb / cockroachdb/cockroach

basalt-26.3: pkg/ccl/testccl/sqlccl/sqlccl_test: TestReadCommittedReadTimestampNotSteppedOnCommit failed

Open
#174,694 3 comments 0 reactions 0 assignees View on GitHub
branch-basalt-26.3 C-test-failure O-robot T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

pkg/ccl/testccl/sqlccl/sqlccl_test.TestReadCommittedReadTimestampNotSteppedOnCommit [failed](https://mesolite.cluster.engflow.com/invocations/default/22e193ae-13db-4b11-9efe-450a9f57dcbf?testReportRun=2&testReportShard=5&testReportAttempt=1#targets-Ly9wa2cvY2NsL3Rlc3RjY2wvc3FsY2NsOnNxbGNjbF90ZXN0) on basalt-26.3 @ [7b7d64a360fe4249bf6409fb266081ab370be355](https://github.com/cockroachdb/cockroach/commits/7b7d64a360fe4249bf6409fb266081ab370be355):

```
read_committed_test.go:250: trace: 2026-09-04 14:38:58.119929 +0000 UTC 00:00:00.021151 r88: sending batch 1 EndTxn to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=36b2cfb1] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 15
read_committed_test.go:250: trace: 2026-09-04 14:38:58.120877 +0000 UTC 00:00:00.022099 querying next range at /Tenant/10/Table/11/3/"\x80"/101/1/"\x01\x01\x80$Zz\x14\xd4\xcdKT\xb32n\x05\xc1\x05\x96\xaa"/0 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=36b2cfb1] kv/kvclient/kvcoord/range_iter.go:183 kv.DistSender: sending pre-commit query intents 16
read_committed_test.go:250: trace: 2026-09-04 14:38:58.121162 +0000 UTC 00:00:00.022384 key: /Tenant/10/Table/11/3/"\x80"/101/1/"\x01\x01\x80$Zz\x14\xd4\xcdKT\xb32n\x05\xc1\x05\x96\xaa"/0, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=36b2cfb1] kv/kvclient/kvcoord/range_iter.go:220 kv.DistSender: sending pre-commit query intents 16
read_committed_test.go:250: trace: 2026-09-04 14:38:58.121306 +0000 UTC 00:00:00.022527 r88: sending batch 1 QueryIntent to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=36b2cfb1] kv/kvclient/kvcoord/dist_sender.go:2806 kv.DistSender: sending pre-commit query intents 16
read_committed_test.go:250: trace: 2026-09-04 14:38:58.143811 +0000 UTC 00:00:00.045033 querying next range at /Tenant/10/NamespaceTable/30/1/100/101/"kv"/4/1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=fbace737] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 24
read_committed_test.go:250: trace: 2026-09-04 14:38:58.144094 +0000 UTC 00:00:00.045316 key: /Tenant/10/NamespaceTable/30/1/100/101/"kv"/4/1, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=fbace737] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 24
read_committed_test.go:250: trace: 2026-09-04 14:38:58.144248 +0000 UTC 00:00:00.045469 r88: sending batch 1 Get to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=fbace737] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 24
read_committed_test.go:250: trace: 2026-09-04 14:38:58.15801 +0000 UTC 00:00:00.059232 querying next range at /Tenant/10/NamespaceTable/30/1/100/101/"kv"/4/1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=766479a0] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 28
read_committed_test.go:250: trace: 2026-09-04 14:38:58.158286 +0000 UTC 00:00:00.059508 key: /Tenant/10/NamespaceTable/30/1/100/101/"kv"/4/1, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=766479a0] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 28
read_committed_test.go:250: trace: 2026-09-04 14:38:58.158437 +0000 UTC 00:00:00.059659 r88: sending batch 1 Get to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=766479a0] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 28
read_committed_test.go:250: trace: 2026-09-04 14:38:58.190953 +0000 UTC 00:00:00.092175 querying next range at /Tenant/10/Table/3/1/104/2/1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 33
read_committed_test.go:250: trace: 2026-09-04 14:38:58.191206 +0000 UTC 00:00:00.092428 key: /Tenant/10/Table/3/1/104/2/1, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 33
read_committed_test.go:250: trace: 2026-09-04 14:38:58.191359 +0000 UTC 00:00:00.092581 r88: sending batch 1 Get to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 33
read_committed_test.go:250: trace: 2026-09-04 14:38:58.198655 +0000 UTC 00:00:00.099876 querying next range at /Tenant/10/Table/11/3/"\x80"/104/1/"\x01\x01\x80$Zz\x14\xd4\xcdKT\xb32n\x05\xc1\x05\x96\xaa"/0 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 37
read_committed_test.go:250: trace: 2026-09-04 14:38:58.198937 +0000 UTC 00:00:00.100159 key: /Tenant/10/Table/11/3/"\x80"/104/1/"\x01\x01\x80$Zz\x14\xd4\xcdKT\xb32n\x05\xc1\x05\x96\xaa"/0, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 37
read_committed_test.go:250: trace: 2026-09-04 14:38:58.199072 +0000 UTC 00:00:00.100294 r88: sending batch 1 CPut to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 37
read_committed_test.go:250: trace: 2026-09-04 14:38:58.206998 +0000 UTC 00:00:00.10822 querying next range at /Tenant/10/Table/11/3/"\x80"/104/1/"\x01\x01\x80$Zz\x14\xd4\xcdKT\xb32n\x05\xc1\x05\x96\xaa"/0 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 42
read_committed_test.go:250: trace: 2026-09-04 14:38:58.207271 +0000 UTC 00:00:00.108493 key: /Tenant/10/Table/11/3/"\x80"/104/1/"\x01\x01\x80$Zz\x14\xd4\xcdKT\xb32n\x05\xc1\x05\x96\xaa"/0, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 42
read_committed_test.go:250: trace: 2026-09-04 14:38:58.207403 +0000 UTC 00:00:00.108625 r88: sending batch 1 EndTxn to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 42
read_committed_test.go:250: trace: 2026-09-04 14:38:58.208297 +0000 UTC 00:00:00.109519 querying next range at /Tenant/10/Table/11/3/"\x80"/104/1/"\x01\x01\x80$Zz\x14\xd4\xcdKT\xb32n\x05\xc1\x05\x96\xaa"/0 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/range_iter.go:183 kv.DistSender: sending pre-commit query intents 43
read_committed_test.go:250: trace: 2026-09-04 14:38:58.208605 +0000 UTC 00:00:00.109827 key: /Tenant/10/Table/11/3/"\x80"/104/1/"\x01\x01\x80$Zz\x14\xd4\xcdKT\xb32n\x05\xc1\x05\x96\xaa"/0, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/range_iter.go:220 kv.DistSender: sending pre-commit query intents 43
read_committed_test.go:250: trace: 2026-09-04 14:38:58.208746 +0000 UTC 00:00:00.109967 r88: sending batch 1 QueryIntent to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=b915d14b] kv/kvclient/kvcoord/dist_sender.go:2806 kv.DistSender: sending pre-commit query intents 43
read_committed_test.go:250: trace: 2026-09-04 14:38:58.229673 +0000 UTC 00:00:00.130895 querying next range at /Tenant/10/Table/20/1/104 [nsql1,client=127.0.0.1:52056,hostssl,user=root,intExec=get-table-statistics] kv/kvclient/kvcoord/range_iter.go:183 sql query 53
read_committed_test.go:250: trace: 2026-09-04 14:38:58.229901 +0000 UTC 00:00:00.131123 key: /Tenant/10/Table/20/1/104, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,intExec=get-table-statistics] kv/kvclient/kvcoord/range_iter.go:220 sql query 53
read_committed_test.go:250: trace: 2026-09-04 14:38:58.231661 +0000 UTC 00:00:00.132883 Scan /Tenant/10/Table/20/1/10{4-5} [nsql1,client=127.0.0.1:52056,hostssl,user=root,intExec=get-table-statistics] sql/row/kv_batch_fetcher.go:665 colbatchscan 58
read_committed_test.go:250: trace: 2026-09-04 14:38:58.231975 +0000 UTC 00:00:00.133196 querying next range at /Tenant/10/Table/20/1/104 [nsql1,client=127.0.0.1:52056,hostssl,user=root,intExec=get-table-statistics,txn=63afb50b] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 60
read_committed_test.go:250: trace: 2026-09-04 14:38:58.232229 +0000 UTC 00:00:00.133451 key: /Tenant/10/Table/20/1/104, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,intExec=get-table-statistics,txn=63afb50b] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 60
read_committed_test.go:250: trace: 2026-09-04 14:38:58.232395 +0000 UTC 00:00:00.133616 r88: sending batch 1 Scan to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,intExec=get-table-statistics,txn=63afb50b] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 60
read_committed_test.go:250: trace: 2026-09-04 14:38:58.264696 +0000 UTC 00:00:00.165918 rows affected: 0 [nsql1,client=127.0.0.1:52056,hostssl,user=root,intExec=get-table-statistics] sql/conn_executor_exec.go:2162 sql query 53
read_committed_test.go:250: trace: 2026-09-04 14:38:58.284322 +0000 UTC 00:00:00.185543 CPut /Tenant/10/Table/104/1/1207377390089273345/0 -> /TUPLE/1:1:Bytes/a/1:2:Int/1 [nsql1,client=127.0.0.1:52056,hostssl,user=root] sql/row/writer.go:308 insert fast path 68
read_committed_test.go:250: trace: 2026-09-04 14:38:58.284981 +0000 UTC 00:00:00.186203 querying next range at /Tenant/10/Table/104/1/1207377390089273345/0 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 70
read_committed_test.go:250: trace: 2026-09-04 14:38:58.285244 +0000 UTC 00:00:00.186465 key: /Tenant/10/Table/104/1/1207377390089273345/0, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 70
read_committed_test.go:250: trace: 2026-09-04 14:38:58.28541 +0000 UTC 00:00:00.186632 r88: sending batch 1 Get to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 70
read_committed_test.go:250: trace: 2026-09-04 14:38:58.322544 +0000 UTC 00:00:00.223765 rows affected: 1 [nsql1,client=127.0.0.1:52056,hostssl,user=root] sql/conn_executor_exec.go:2884 sql query 2
read_committed_test.go:250: trace: 2026-09-04 14:38:58.38531 +0000 UTC 00:00:00.286532 CPut /Tenant/10/Table/104/1/1207377390420230145/0 -> /TUPLE/1:1:Bytes/b/1:2:Int/2 [nsql1,client=127.0.0.1:52056,hostssl,user=root] sql/row/writer.go:308 insert fast path 79
read_committed_test.go:250: trace: 2026-09-04 14:38:58.385803 +0000 UTC 00:00:00.287025 querying next range at /Tenant/10/Table/104/1/1207377390420230145/0 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 81
read_committed_test.go:250: trace: 2026-09-04 14:38:58.386069 +0000 UTC 00:00:00.287291 key: /Tenant/10/Table/104/1/1207377390420230145/0, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 81
read_committed_test.go:250: trace: 2026-09-04 14:38:58.386243 +0000 UTC 00:00:00.287464 r88: sending batch 1 Get to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 81
read_committed_test.go:250: trace: 2026-09-04 14:38:58.51512 +0000 UTC 00:00:00.416342 rows affected: 1 [nsql1,client=127.0.0.1:52056,hostssl,user=root] sql/conn_executor_exec.go:2884 sql query 73
read_committed_test.go:250: trace: 2026-09-04 14:38:58.578724 +0000 UTC 00:00:00.479946 CPut /Tenant/10/Table/104/1/1207377391053996033/0 -> /TUPLE/1:1:Bytes/c/1:2:Int/3 [nsql1,client=127.0.0.1:52056,hostssl,user=root] sql/row/writer.go:308 insert fast path 91
read_committed_test.go:250: trace: 2026-09-04 14:38:58.579262 +0000 UTC 00:00:00.480484 querying next range at /Tenant/10/Table/104/1/1207377391053996033/0 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 93
read_committed_test.go:250: trace: 2026-09-04 14:38:58.57953 +0000 UTC 00:00:00.480751 key: /Tenant/10/Table/104/1/1207377391053996033/0, desc: r88:/Tenant/1{0-1} [(n1,s1):1, next=2, gen=2, sticky=1788536331.752235575,0] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 93
read_committed_test.go:250: trace: 2026-09-04 14:38:58.579713 +0000 UTC 00:00:00.480934 r88: sending batch 1 Get to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 93
read_committed_test.go:250: trace: 2026-09-04 14:38:58.685181 +0000 UTC 00:00:00.586403 rows affected: 1 [nsql1,client=127.0.0.1:52056,hostssl,user=root] sql/conn_executor_exec.go:2884 sql query 85
read_committed_test.go:250: trace: 2026-09-04 14:38:58.749201 +0000 UTC 00:00:00.650422 querying next range at /Tenant/10/Table/104/1/1207377390089273345/0 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/range_iter.go:183 dist sender send 99
read_committed_test.go:250: trace: 2026-09-04 14:38:58.749492 +0000 UTC 00:00:00.650714 key: /Tenant/10/Table/104/1/1207377390089273345/0, desc: r90:/Tenant/1{0/Table/11-1} [(n1,s1):1, next=2, gen=3] [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/range_iter.go:220 dist sender send 99
read_committed_test.go:250: trace: 2026-09-04 14:38:58.749681 +0000 UTC 00:00:00.650903 r90: sending batch 3 Put, 1 EndTxn to (n1,s1):1 [nsql1,client=127.0.0.1:52056,hostssl,user=root,txn=4cec7ecb] kv/kvclient/kvcoord/dist_sender.go:2806 dist sender send 99
panic.go:694: -- test log scope end --
test logs left over in: outputs.zip/logTestReadCommittedReadTimestampNotSteppedOnCommit3211734318
--- FAIL: TestReadCommittedReadTimestampNotSteppedOnCommit (17.71s)
```

Parameters:
- attempt=1
- race=true
- run=2
- shard=5
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestReadCommittedReadTimestampNotSteppedOnCommit.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-67913

Contributor guide

Open the contributing guide

Research direction

Start with `pkg/ccl/testccl/sqlccl/sqlccl_test` and the failing `TestReadCommittedReadTimestampNotSteppedOnCommit` test; the trace points to `read_committed_test.go:250`. Reproduce the failure on the cited basalt-26.3 commit and inspect the test’s expectations and trace. Done means identifying and fixing the cause so the test passes reliably; the issue provides no specific expected change.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.