cockroachdb / cockroachdb/cockroach

basalt-26.3.2-rc-20260914: sql/stats: TestCacheAutoRefresh failed

Open
#175,669 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

branch-basalt-26.3.2-rc-20260914 C-test-failure O-robot release-blocker T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

sql/stats.TestCacheAutoRefresh failed on basalt-26.3.2-rc-20260914 @ d3eb08ebdd1dacc8d3dac0b107a7db5929e09969:


goroutine 415891 [select]:
github.com/cockroachdb/cockroach/pkg/util/admission.(*WorkQueue).Admit(0xc001aa3cc0, {0xa4e9ce8, 0xc045cffc28}, {{0x1}, 0x0, 0x0, 0x18d6b22a5f6af80f, 0x0, 0x1, {0x0, ...}, ...})
	pkg/util/admission/work_queue.go:1104 +0x2405
github.com/cockroachdb/cockroach/pkg/kv/kvserver/kvadmission.(*controllerImpl).AdmitKVWork(0xc003680840, {0xa4e9ce8, 0xc045cffc28}, {0x0?}, {0x0?}, 0xc02cb39a20)
	pkg/kv/kvserver/kvadmission/kvadmission.go:412 +0xdcf
github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal(0xc0020d3808, {0xa4e9ce8, 0xc045cffc28}, {0x3?}, 0xc02cb39a20)
	pkg/server/node.go:1688 +0x7c2
github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch(0xc0020d3808, {0xa4e8330, 0xc04b567410}, 0xc02cb39a20)
	pkg/server/node.go:1929 +0x3f6
github.com/cockroachdb/cockroach/pkg/kv/kvpb._Internal_Batch_Handler.func1({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20})
	bazel-out/k8-fastbuild/bin/pkg/kv/kvpb/kvpb_go_proto_/github.com/cockroachdb/cockroach/pkg/kv/kvpb/api.pb.go:12762 +0xbf
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.ServerInterceptor.func12({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20}, 0xc01a1145e0, 0xc0478d1a88)
	pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:60 +0x74c
google.golang.org/grpc.getChainUnaryHandler.func1({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20})
	external/org_golang_google_grpc/server.go:1177 +0x23d
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20}, 0xc01a1145e0, 0xc027cc7140)
	pkg/rpc/context.go:182 +0xf1
google.golang.org/grpc.getChainUnaryHandler.func1.getChainUnaryHandler.1({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20})
	external/org_golang_google_grpc/server.go:1177 +0x11f
github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.unaryInterceptor({0xc0021a0000, {{0x1}, {0xa547e40, 0xc00231e780}}, 0x0}, {0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20}, 0xc01a1145e0, ...)
	pkg/rpc/auth.go:96 +0x325
google.golang.org/grpc.getChainUnaryHandler.func1({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20})
	external/org_golang_google_grpc/server.go:1177 +0x23d
github.com/cockroachdb/cockroach/pkg/rpc.gatewayRequestRecoveryInterceptor({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20}, 0xc01a1145e0?, 0xc027cc70c0)
	pkg/rpc/metrics.go:803 +0x183
google.golang.org/grpc.getChainUnaryHandler.func1.getChainUnaryHandler.1({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20})
	external/org_golang_google_grpc/server.go:1177 +0x11f
github.com/cockroachdb/cockroach/pkg/server.newGRPCServer.NewRequestMetricsInterceptor.func4({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20}, 0xc01a1145e0, 0xc027cc7080)
	pkg/rpc/metrics.go:589 +0x3a2
google.golang.org/grpc.getChainUnaryHandler.func1({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20})
	external/org_golang_google_grpc/server.go:1177 +0x23d
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1({0xa4e8330, 0xc04b567410})
	pkg/rpc/context.go:139 +0x6b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc001c1f5e0, {0xa4e8330, 0xc04b567410}, {0xa0eed45, 0x21}, 0xc0463ed6c8)
	pkg/util/stop/stopper.go:358 +0x237
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20}, 0xc01a1145e0, 0xc027cc7040)
	pkg/rpc/context.go:137 +0x14c
google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1({0xa4e8330, 0xc04b567410}, {0x9fa2200, 0xc02cb39a20}, 0xc01a1145e0, 0xc0478d1a88)
	external/org_golang_google_grpc/server.go:1168 +0xe8
github.com/cockroachdb/cockroach/pkg/kv/kvpb._Internal_Batch_Handler({0x9f94480, 0xc0020d3808}, {0xa4e8330, 0xc04b567410}, 0xc0318d0fc0, 0xc00367b160)
	bazel-out/k8-fastbuild/bin/pkg/kv/kvpb/kvpb_go_proto_/github.com/cockroachdb/cockroach/pkg/kv/kvpb/api.pb.go:12764 +0x1e7
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0012f9a40, {0xa5490a0, 0xc0186be1a0}, 0xc029040b40, 0xc0018af050, 0x10920700, 0x0)
	external/org_golang_google_grpc/server.go:1358 +0x1871
google.golang.org/grpc.(*Server).handleStream(0xc0012f9a40, {0xa5490a0, 0xc0186be1a0}, 0xc029040b40, 0x0)
	external/org_golang_google_grpc/server.go:1735 +0xff3
google.golang.org/grpc.(*Server).serveStreams.func1.1()
	external/org_golang_google_grpc/server.go:970 +0x173
created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 6696
	external/org_golang_google_grpc/server.go:981 +0x253

Parameters:

  • attempt=1
  • race=true
  • run=2
  • shard=8
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • #175423 basalt-26.3: sql/stats: TestCacheAutoRefresh failed [C-test-failure O-robot T-sql-queries branch-basalt-26.3]
  • #175419 basalt-26.3.2-rc-20260913: sql/stats: TestCacheAutoRefresh failed [C-test-failure O-robot T-sql-queries branch-basalt-26.3.2-rc-20260913]
  • #174912 basalt-26.3.2-rc-20260908: sql/stats: TestCacheAutoRefresh failed [C-test-failure O-robot T-sql-queries branch-basalt-26.3.2-rc-20260908]

This test on roachdash | Improve this report!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the sql/stats.TestCacheAutoRefresh failure and its linked test report, then trace the stack through pkg/util/admission/work_queue.go:1104 and pkg/kv/kvserver/kvadmission/kvadmission.go:412. Investigate why the test stalls or fails under race mode and confirm the test passes reliably on the affected basalt branches.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
backend, databases, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.