cockroachdb / cockroachdb/cockroach
crosscluster/physical: TestIngestSpanConfigsFullScan failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
crosscluster/physical.TestIngestSpanConfigsFullScan [failed](https://mesolite.cluster.engflow.com/invocations/default/62c82f29-a7d2-4286-b985-c719f1445695?testReportRun=1&testReportShard=14&testReportAttempt=1#targets-Ly9wa2cvY3Jvc3NjbHVzdGVyL3BoeXNpY2FsOnBoeXNpY2FsX3Rlc3Q=) on staging-v25.4.16 @ [65ac8a6bd3d10290de3f25297aaa6d1c3de3c369](https://github.com/cockroachdb/cockroach/commits/65ac8a6bd3d10290de3f25297aaa6d1c3de3c369):
```
crypto/tls.(*Conn).readRecordOrCCS(0xc0094c0708, 0x0)
GOROOT/src/crypto/tls/conn.go:629 +0x3cf
crypto/tls.(*Conn).readRecord(...)
GOROOT/src/crypto/tls/conn.go:591
crypto/tls.(*Conn).Read(0xc0094c0708, {0xc0096a0000, 0x8000, 0xc006c75cf0?})
GOROOT/src/crypto/tls/conn.go:1385 +0x150
bufio.(*Reader).Read(0xc0095d3ce0, {0xc00302e2e0, 0x9, 0x0?})
GOROOT/src/bufio/bufio.go:241 +0x197
io.ReadAtLeast({0x88090e0, 0xc0095d3ce0}, {0xc00302e2e0, 0x9, 0x9}, 0x9)
GOROOT/src/io/io.go:335 +0x90
io.ReadFull(...)
GOROOT/src/io/io.go:354
golang.org/x/net/http2.readFrameHeader({0xc00302e2e0, 0x9, 0xc0093d2bd0?}, {0x88090e0?, 0xc0095d3ce0?})
external/org_golang_x_net/http2/frame.go:242 +0x65
golang.org/x/net/http2.(*Framer).ReadFrame(0xc00302e2a0)
external/org_golang_x_net/http2/frame.go:506 +0x85
google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc0095b4488, 0xc0095b75e0)
external/org_golang_google_grpc/internal/transport/http2_client.go:1595 +0x226
created by google.golang.org/grpc/internal/transport.newHTTP2Client in goroutine 7365
external/org_golang_google_grpc/internal/transport/http2_client.go:397 +0x1d33
goroutine 7341 [select]:
google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc0092ab040)
external/org_golang_google_grpc/internal/transport/http2_server.go:1150 +0x1f3
created by google.golang.org/grpc/internal/transport.NewServerTransport in goroutine 7336
external/org_golang_google_grpc/internal/transport/http2_server.go:339 +0x1a8e
goroutine 7339 [select]:
google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc00944afa0, 0x1)
external/org_golang_google_grpc/internal/transport/controlbuf.go:418 +0x113
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0094fefc0)
external/org_golang_google_grpc/internal/transport/controlbuf.go:552 +0x7b
google.golang.org/grpc/internal/transport.newHTTP2Client.func6()
external/org_golang_google_grpc/internal/transport/http2_client.go:451 +0x85
created by google.golang.org/grpc/internal/transport.newHTTP2Client in goroutine 7334
external/org_golang_google_grpc/internal/transport/http2_client.go:449 +0x22db
goroutine 87160 [select]:
github.com/cockroachdb/cockroach/pkg/sql/stats.(*Refresher).Start.func1.1({0x886fa88, 0xc006a12960})
pkg/sql/stats/automatic_stats.go:625 +0x1b5
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0x886fa88, 0xc006a12960})
pkg/util/stop/stopper.go:447 +0x8a
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1965
pkg/util/stop/stopper.go:445 +0x137
goroutine 78122 [select]:
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker.func1()
external/com_github_cockroachdb_pebble/vfs/disk_health.go:254 +0xb3
created by github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker in goroutine 508
external/com_github_cockroachdb_pebble/vfs/disk_health.go:249 +0x58
```
Parameters:
- attempt=1
- run=1
- shard=14
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
/cc @cockroachlabs/unowned
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestIngestSpanConfigsFullScan.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Contributor guide
Research direction
Start with the linked Mesolite test report and the crosscluster/physical.TestIngestSpanConfigsFullScan failure on staging-v25.4.16; inspect the reported Go TLS/gRPC goroutine stack and the referenced commit. Reproduce the failure and identify its cause, with the test passing reliably as the done condition.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, grpc
- Domain
- databases, distributed-systems, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100