cockroachdb / cockroachdb/cockroach
ccl/multiregionccl: TestRegionLivenessProberForLeases failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
ccl/multiregionccl.TestRegionLivenessProberForLeases [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Ci_TestsAwsLinuxArm64_UnitTests/21440922?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Ci_TestsAwsLinuxArm64_UnitTests/21440922?buildTab=artifacts#/) on master @ [dbfee5672e06e6a455a79d70f7ad46a47f6707ac](https://github.com/cockroachdb/cockroach/commits/dbfee5672e06e6a455a79d70f7ad46a47f6707ac):
```
goroutine 44211 [select, 14 minutes, "tags":"n1,rnode=2,raddr=127.0.0.1:37353,class=default,rpc", "workload.id":"RPC_HEARTBEAT"]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x31a1230daf40, {0x712d910, 0x31a1234b74f0})
external/org_golang_google_grpc/internal/grpcsync/callback_serializer.go:83 +0xc0
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 44210
external/org_golang_google_grpc/internal/grpcsync/callback_serializer.go:55 +0x10c
goroutine 39066 [select, 14 minutes]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Tracer).runPeriodicSnapshotsLoop(0x31a0fa9e5ef8?, 0x31a0f9698000, 0x31a1023aa700, 0x31a13269b490, 0x70cc620?)
pkg/util/tracing/tracer_snapshots.go:279 +0xbc
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Tracer).PeriodicSnapshotsLoop(0x31a11a5c4b40, 0x31a0f9698000, 0x31a13269b490)
pkg/util/tracing/tracer_snapshots.go:245 +0xb0
github.com/cockroachdb/cockroach/pkg/server.newSQLServer.func1({0x31a126322540?, 0x71306e8?})
pkg/server/server_sql.go:575 +0x44
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0x71306e8, 0x31a1224866e0})
pkg/util/stop/stopper.go:447 +0x7c
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 39410
pkg/util/stop/stopper.go:445 +0xe0
goroutine 44827 [select, 14 minutes, "n":"3", "replicate":""]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2({0x712d948, 0x31a12124b620})
pkg/kv/kvserver/queue.go:1131 +0xbc
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0x712d948, 0x31a12124b620})
pkg/util/stop/stopper.go:447 +0x7c
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 44590
pkg/util/stop/stopper.go:445 +0xe0
goroutine 44829 [select, "n":"3", "raftlog":""]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2({0x712d948, 0x31a12124b6e0})
pkg/kv/kvserver/queue.go:1131 +0xbc
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0x712d948, 0x31a12124b6e0})
pkg/util/stop/stopper.go:447 +0x7c
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 44590
pkg/util/stop/stopper.go:445 +0xe0
goroutine 44830 [select, 14 minutes, "n":"3", "raftsnapshot":""]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2({0x712d948, 0x31a12124b710})
pkg/kv/kvserver/queue.go:1131 +0xbc
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0x712d948, 0x31a12124b710})
pkg/util/stop/stopper.go:447 +0x7c
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 44590
pkg/util/stop/stopper.go:445 +0xe0
goroutine 44828 [select, 10 minutes, "n":"3", "replicaGC":""]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2({0x712d948, 0x31a12124b650})
pkg/kv/kvserver/queue.go:1131 +0xbc
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0x712d948, 0x31a12124b650})
pkg/util/stop/stopper.go:447 +0x7c
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 44590
pkg/util/stop/stopper.go:445 +0xe0
```
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestRegionLivenessProberForLeases.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-66111
Contributor guide
Research direction
Start with the TeamCity failure log and artifacts for ccl/multiregionccl.TestRegionLivenessProberForLeases at commit dbfee5672e06e6a455a79d70f7ad46a47f6707ac. Inspect the reported goroutine stacks and the test entry point to determine why the test hangs or fails. Done means identifying the failure cause, addressing it, and confirming the test passes reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100