cockroachdb / cockroachdb/cockroach
release-26.2: util/startup: TestStartupFailureRandomRange failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
util/startup.TestStartupFailureRandomRange [failed](https://mesolite.cluster.engflow.com/invocations/default/242a15b3-a450-4414-8072-ba10470dd1b0?testReportRun=3&testReportShard=1&testReportAttempt=1#targets-Ly9wa2cvdXRpbC9zdGFydHVwOnN0YXJ0dXBfdGVzdA==) on release-26.2 @ [914fcaa1a2da846f700b172e18381b6763054f37](https://github.com/cockroachdb/cockroach/commits/914fcaa1a2da846f700b172e18381b6763054f37):
```
github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.streamInterceptor({0xc003d5a000, {{0x1}, {0x6f90400, 0xc0012cd860}}, 0x0}, {0x6125bc0, 0xc004067c08}, {0x6f6aff0, 0xc02b79b4a0}, 0xc04e53b548, ...)
pkg/rpc/auth.go:192 +0x3a9
google.golang.org/grpc.getChainStreamHandler.func1({0x6125bc0?, 0xc004067c08?}, {0x6f6aff0?, 0xc02b79b4a0?})
external/org_golang_google_grpc/server.go:1504 +0x153
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func2.1({0xc00326b170?, 0x6f33838?})
pkg/rpc/context.go:148 +0x27
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc00326b170, {0x6f33838, 0xc030ca7800}, {0xc03089c7b0, 0x27}, 0xc060b6aab0)
pkg/util/stop/stopper.go:358 +0x137
github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func2({0x6125bc0, 0xc004067c08}, {0x6f6aff0, 0xc02b79b4a0}, 0xc04e53b548, 0xc020142bc0)
pkg/rpc/context.go:147 +0xcc
google.golang.org/grpc.NewServer.chainStreamServerInterceptors.chainStreamInterceptors.func2({0x6125bc0, 0xc004067c08}, {0x6f6aff0, 0xc02b79b4a0}, 0xc04e53b548, 0xc01fbe0860?)
external/org_golang_google_grpc/server.go:1495 +0x7c
google.golang.org/grpc.(*Server).processStreamingRPC(0xc0037921e0, {0x6f913c0, 0xc013b47ba0}, 0xc025670a20, 0xc0011f7380, 0xac756a0, 0x0)
external/org_golang_google_grpc/server.go:1659 +0x1265
google.golang.org/grpc.(*Server).handleStream(0xc0037921e0, {0x6f913c0, 0xc013b47ba0}, 0xc025670a20, 0x0)
external/org_golang_google_grpc/server.go:1739 +0x987
google.golang.org/grpc.(*Server).serveStreams.func1.1()
external/org_golang_google_grpc/server.go:970 +0xaf
created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 104084067
external/org_golang_google_grpc/server.go:981 +0x133
goroutine 106985374 [runnable]:
github.com/sasha-s/go-deadlock.lock.func1()
external/com_github_sasha_s_go_deadlock/deadlock.go:182
created by github.com/sasha-s/go-deadlock.lock in goroutine 26772793
external/com_github_sasha_s_go_deadlock/deadlock.go:182 +0x151
goroutine 106899694 [runnable]:
github.com/sasha-s/go-deadlock.lock.func1()
external/com_github_sasha_s_go_deadlock/deadlock.go:182
created by github.com/sasha-s/go-deadlock.lock in goroutine 327647
external/com_github_sasha_s_go_deadlock/deadlock.go:182 +0x151
goroutine 106984194 [runnable]:
github.com/sasha-s/go-deadlock.lock.func1()
external/com_github_sasha_s_go_deadlock/deadlock.go:182
created by github.com/sasha-s/go-deadlock.lock in goroutine 26772793
external/com_github_sasha_s_go_deadlock/deadlock.go:182 +0x151
goroutine 106903207 [runnable]:
github.com/sasha-s/go-deadlock.lock.func1()
external/com_github_sasha_s_go_deadlock/deadlock.go:182
created by github.com/sasha-s/go-deadlock.lock in goroutine 71
external/com_github_sasha_s_go_deadlock/deadlock.go:182 +0x151
goroutine 106988280 [runnable]:
github.com/sasha-s/go-deadlock.lock.func1()
external/com_github_sasha_s_go_deadlock/deadlock.go:182
created by github.com/sasha-s/go-deadlock.lock in goroutine 105358109
external/com_github_sasha_s_go_deadlock/deadlock.go:182 +0x151
```
Parameters:
- attempt=1
- deadlock=true
- run=3
- shard=1
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
Same failure on other branches
- #172692 util/startup: TestStartupFailureRandomRange failed [C-test-failure O-robot branch-master release-blocker]
/cc @cockroachlabs/unowned
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestStartupFailureRandomRange.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-66047
Contributor guide
Research direction
Start with util/startup.TestStartupFailureRandomRange and reproduce the release-26.2 failure using the linked invocation details. Read the referenced internal guide and inspect the goroutine stack, including pkg/rpc/auth.go:192 and pkg/util/stop/stopper.go:358; done means the intermittent failure is explained and the relevant test or startup behavior is verified across the reported branches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100