cockroachdb / cockroachdb/cockroach
basalt-26.3.2-rc-20260916: sql/stats: TestCacheAutoRefresh failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
sql/stats.TestCacheAutoRefresh [failed](https://mesolite.cluster.engflow.com/invocations/default/9f7a7420-cbb9-454a-8939-99ee51e4cf97?testReportRun=1&testReportShard=8&testReportAttempt=1#targets-Ly9wa2cvc3FsL3N0YXRzOnN0YXRzX3Rlc3Q=) on basalt-26.3.2-rc-20260916 @ [20b5b7762461dc7d59206ab797cd12d09c877eaa](https://github.com/cockroachdb/cockroach/commits/20b5b7762461dc7d59206ab797cd12d09c877eaa):
```
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).SendWithWorkStats(0xc0037f9008, {0xa4ec6c8, 0xc0302b2640}, 0xc014dc6840, 0xc0320581c0, {0x0?})
pkg/kv/kvserver/replica_send.go:196 +0x92d
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).SendWithWorkStats(0xc0060e0008, {0xa4ec6c8, 0xc0302b25a8}, 0xc014dc6840, 0xc0320581c0, {0xc02a6a82b0?})
pkg/kv/kvserver/store_send.go:242 +0xc65
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).SendWithWorkStats(0xc00104f0e0, {0xa4ec6c8, 0xc0302b25a8}, 0xc014dc6840, 0xc0320581c0, {0x0?})
pkg/kv/kvserver/stores.go:195 +0xfc
github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal(0xc001d63008, {0xa4ec6c8, 0xc0302b25a8}, {0x3?}, 0xc014dc6840)
pkg/server/node.go:1736 +0xbc5
github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch(0xc001d63008, {0xa4ead10, 0xc010f4d770}, 0xc014dc6840)
pkg/server/node.go:1929 +0x3f6
github.com/cockroachdb/cockroach/pkg/kv/kvpb._Internal_Batch_Handler.func1({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840})
bazel-out/k8-fastbuild/bin/pkg/kv/kvpb/kvpb_go_proto_/github.com/cockroachdb/cockroach/pkg/kv/kvpb/api.pb.go:12762 +0xbf
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.ServerInterceptor.func12({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840}, 0xc02e064f20, 0xc03850b638)
pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:60 +0x74c
google.golang.org/grpc.getChainUnaryHandler.func1({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840})
external/org_golang_google_grpc/server.go:1177 +0x23d
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840}, 0xc02e064f20, 0xc031cd75c0)
pkg/rpc/context.go:182 +0xf1
google.golang.org/grpc.getChainUnaryHandler.func1.getChainUnaryHandler.1({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840})
external/org_golang_google_grpc/server.go:1177 +0x11f
github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.unaryInterceptor({0xc00211a000, {{0x1}, {0xa54a820, 0xc00230bce0}}, 0x0}, {0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840}, 0xc02e064f20, ...)
pkg/rpc/auth.go:96 +0x325
google.golang.org/grpc.getChainUnaryHandler.func1({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840})
external/org_golang_google_grpc/server.go:1177 +0x23d
github.com/cockroachdb/cockroach/pkg/rpc.gatewayRequestRecoveryInterceptor({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840}, 0xc02e064f20?, 0xc031cd7540)
pkg/rpc/metrics.go:803 +0x183
google.golang.org/grpc.getChainUnaryHandler.func1.getChainUnaryHandler.1({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840})
external/org_golang_google_grpc/server.go:1177 +0x11f
github.com/cockroachdb/cockroach/pkg/server.newGRPCServer.NewRequestMetricsInterceptor.func4({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840}, 0xc02e064f20, 0xc031cd7500)
pkg/rpc/metrics.go:589 +0x3a2
google.golang.org/grpc.getChainUnaryHandler.func1({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840})
external/org_golang_google_grpc/server.go:1177 +0x23d
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1({0xa4ead10, 0xc010f4d770})
pkg/rpc/context.go:139 +0x6b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc001c8f5e0, {0xa4ead10, 0xc010f4d770}, {0xa0f1545, 0x21}, 0xc02a6a96c8)
pkg/util/stop/stopper.go:358 +0x237
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840}, 0xc02e064f20, 0xc031cd74c0)
pkg/rpc/context.go:137 +0x14c
google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1({0xa4ead10, 0xc010f4d770}, {0x9fa4a00, 0xc014dc6840}, 0xc02e064f20, 0xc03850b638)
external/org_golang_google_grpc/server.go:1168 +0xe8
github.com/cockroachdb/cockroach/pkg/kv/kvpb._Internal_Batch_Handler({0x9f96c80, 0xc001d63008}, {0xa4ead10, 0xc010f4d770}, 0xc02ce865b0, 0xc0035d0280)
bazel-out/k8-fastbuild/bin/pkg/kv/kvpb/kvpb_go_proto_/github.com/cockroachdb/cockroach/pkg/kv/kvpb/api.pb.go:12764 +0x1e7
google.golang.org/grpc.(*Server).processUnaryRPC(0xc00247cd20, {0xa54ba80, 0xc003286000}, 0xc0161447e0, 0xc001d09b60, 0x10926700, 0x0)
external/org_golang_google_grpc/server.go:1358 +0x1871
google.golang.org/grpc.(*Server).handleStream(0xc00247cd20, {0xa54ba80, 0xc003286000}, 0xc0161447e0, 0x0)
external/org_golang_google_grpc/server.go:1735 +0xff3
google.golang.org/grpc.(*Server).serveStreams.func1.1()
external/org_golang_google_grpc/server.go:970 +0x173
created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 6023
external/org_golang_google_grpc/server.go:981 +0x253
```
Parameters:
- attempt=1
- race=true
- run=1
- shard=8
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
Same failure on other branches
- #175423 basalt-26.3: sql/stats: TestCacheAutoRefresh failed [C-test-failure O-robot T-sql-queries branch-basalt-26.3]
- #175419 basalt-26.3.2-rc-20260913: sql/stats: TestCacheAutoRefresh failed [C-test-failure O-robot T-sql-queries branch-basalt-26.3.2-rc-20260913]
- #174912 basalt-26.3.2-rc-20260908: sql/stats: TestCacheAutoRefresh failed [C-test-failure O-robot T-sql-queries branch-basalt-26.3.2-rc-20260908]
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestCacheAutoRefresh.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Contributor guide
Research direction
Start by reproducing sql/stats.TestCacheAutoRefresh with race detection using the linked invocation details, then inspect the reported call path beginning at pkg/kv/kvserver/replica_send.go:196 and the related store and node send functions. Done means the test no longer fails on the affected basalt branches, with the underlying failure explained and covered by the test run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100