cockroachdb / cockroachdb/cockroach
release-26.3: ccl/changefeedccl: TestChangefeedCreateKafkaTopicsExplicit failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
ccl/changefeedccl.TestChangefeedCreateKafkaTopicsExplicit [failed](https://mesolite.cluster.engflow.com/invocations/default/c6200add-c7a3-4d2d-bd33-d68b8bfc6600?testReportRun=9&testReportShard=4&testReportAttempt=1#targets-Ly9wa2cvY2NsL2NoYW5nZWZlZWRjY2w6Y2hhbmdlZmVlZGNjbF90ZXN0) on release-26.3 @ [ac78318a5aee72c7e0dbc7ddb5f0c13897a70451](https://github.com/cockroachdb/cockroach/commits/ac78318a5aee72c7e0dbc7ddb5f0c13897a70451):
Fatal error:
```
panic: found 1 short-living non-stopped monitors in flow-62ef6387
(1) attached stack trace
-- stack trace:
| runtime.gopanic
| GOROOT/src/runtime/panic.go:860
| github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait
| pkg/util/ctxgroup/ctxgroup.go:151
| github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GoAndWait
| pkg/util/ctxgroup/ctxgroup.go:224
| github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.startDistChangefeed
| pkg/ccl/changefeedccl/changefeed_dist.go:339
| github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changefeedResumer).resumeWithRetries.func3
| pkg/ccl/changefeedccl/changefeed_stmt.go:2026
| github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changefeedResumer).resumeWithRetries.Group.GoCtx.func6
| pkg/util/ctxgroup/ctxgroup.go:200
| golang.org/x/sync/errgroup.(*Group).Go.func1
| external/org_golang_x_sync/errgroup/errgroup.go:93
Wraps: (2) attached stack trace
-- stack trace:
| runtime.gopanic
| GOROOT/src/runtime/panic.go:860
| [...repeated from below...]
Wraps: (3) assertion failure
Wraps: (4) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).doStop
| pkg/util/mon/bytes_usage.go:781
| github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).Stop
| pkg/util/mon/bytes_usage.go:722
| github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Cleanup
| pkg/sql/flowinfra/flow.go:729
| github.com/cockroachdb/cockroach/pkg/sql/rowflow.(*rowBasedFlow).Cleanup
| pkg/sql/rowflow/row_based_flow.go:490
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run.func3
| pkg/sql/distsql_running.go:1026
| runtime.gopanic
| GOROOT/src/runtime/panic.go:860
| github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait
| pkg/util/ctxgroup/ctxgroup.go:151
| github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).Close
| pkg/ccl/changefeedccl/batching_sink.go:248
| github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.errorWrapperSink.Close
| pkg/ccl/changefeedccl/sink.go:445
| github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeFrontier).close
| pkg/ccl/changefeedccl/changefeed_processors.go:1577
| github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeFrontier).ConsumerClosed
| pkg/ccl/changefeedccl/changefeed_processors.go:2424
| github.com/cockroachdb/cockroach/pkg/sql/execinfra.Run
| pkg/sql/execinfra/base.go:208
| github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run
| pkg/sql/execinfra/processorsbase.go:781
| github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Run
| pkg/sql/flowinfra/flow.go:593
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run
| pkg/sql/distsql_running.go:1091
| github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.startDistChangefeed.func1
| pkg/ccl/changefeedccl/changefeed_dist.go:335
| github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GoAndWait.Group.GoCtx.func1
| pkg/util/ctxgroup/ctxgroup.go:200
| golang.org/x/sync/errgroup.(*Group).Go.func1
| external/org_golang_x_sync/errgroup/errgroup.go:93
| runtime.goexit
| src/runtime/asm_amd64.s:1771
Wraps: (5) found 1 short-living non-stopped monitors in flow-62ef6387
| flow-62ef6387 0 B
| changefntr-mem 0 B
Error types: (1) *withstack.withStack (2) *withstack.withStack (3) *assert.withAssertionFailure (4) *withstack.withStack (5) *errutil.leafError
```
Stack:
```
goroutine 116510 [running]:
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait({0xdef6ddcf540?, {0x823b5f8?, 0xdef6b8e9810?}, 0xdef6c532f00?})
pkg/util/ctxgroup/ctxgroup.go:151 +0xbb
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changefeedResumer).resumeWithRetries(0xdef6ea101b0, {0x823b630, 0xdef6ddd4a20}, {0x82b09c0, 0xdef6a21fc58}, 0x10bc0b2f0a080001, {0xdef6e174720, {0xdef5d34f950, 0x28}, 0xdef6e1746f0, ...}, ...)
pkg/ccl/changefeedccl/changefeed_stmt.go:2061 +0x14f4
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changefeedResumer).Resume(0xdef6ea101b0, {0x823b630, 0xdef6ddd4a20}, {0x78a08e0?, 0xdef6a21fc58})
pkg/ccl/changefeedccl/changefeed_stmt.go:1692 +0x34c
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine.func2(0xdef6f5b1b00?, 0x0?, 0xdef45a1b8c0, {0x820e970?, 0xdef6ea101b0?}, {0x823b630?, 0xdef6ddd4a20?}, {0x78a08e0?, 0xdef6a21fc58?})
pkg/jobs/registry.go:1690 +0xb4
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine(0xdef4713ea88, {0x823dd50, 0xdef6eefbd60}, {0x78a08e0, 0xdef6a21fc58}, {0x820e970, 0xdef6ea101b0}, 0xdef6f5b1b00, {0xdef68a1f040, 0x7}, ...)
pkg/jobs/registry.go:1691 +0xb7f
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).runJob(0xdef4713ea88, {0x823dd50, 0xdef6eefbd60}, {0x820e970, 0xdef6ea101b0}, 0xdef6f5b1b00, {0xdef68a1f040, 0x7}, {0xdef6db336b0, 0x17})
pkg/jobs/adopt.go:503 +0x585
github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).resumeJob.func1({0x823dd50?, 0xdef6eefaa00?})
pkg/jobs/adopt.go:350 +0x13c
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0x823dd50, 0xdef6eefaa00})
pkg/util/stop/stopper.go:447 +0x8a
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 116515
pkg/util/stop/stopper.go:445 +0x125
```
Log preceding fatal error
```
=== RUN TestChangefeedCreateKafkaTopicsExplicit
test_log_scope.go:172: test logs captured to: outputs.zip/logTestChangefeedCreateKafkaTopicsExplicit2480316584
test_log_scope.go:82: use -show-logs to present logs inline
helpers_test.go:1578: making server as system tenant
```
Parameters:
- attempt=1
- run=9
- shard=4
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
/cc @cockroachlabs/cdc
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestChangefeedCreateKafkaTopicsExplicit.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-67649
Contributor guide
Assessment
This issue has not been assessed yet.