cockroachdb / cockroachdb/cockroach
release-26.2.6-rc: pkg/ccl/telemetryccl/telemetryccl_test: TestBulkJobTelemetryLogging failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
pkg/ccl/telemetryccl/telemetryccl_test.TestBulkJobTelemetryLogging [failed](https://mesolite.cluster.engflow.com/invocations/default/e42064ff-182f-4cd0-a88b-ba300022f575?testReportRun=2&testReportShard=2&testReportAttempt=1#targets-Ly9wa2cvY2NsL3RlbGVtZXRyeWNjbDp0ZWxlbWV0cnljY2xfdGVzdA==) on release-26.2.6-rc @ [f0adfbe4c645d4d458de930b415fd46c4459ad90](https://github.com/cockroachdb/cockroach/commits/f0adfbe4c645d4d458de930b415fd46c4459ad90):
```
pkg/util/metric/registry.go:265 +0x103
github.com/cockroachdb/cockroach/pkg/util/metric.(*Registry).Each()
pkg/util/metric/registry.go:265 +0x103
github.com/cockroachdb/cockroach/pkg/util/metric.(*Registry).Each()
pkg/util/metric/registry.go:265 +0x103
github.com/cockroachdb/cockroach/pkg/util/metric.(*Registry).Each()
pkg/util/metric/registry.go:265 +0x103
github.com/cockroachdb/cockroach/pkg/util/metric.(*Registry).Each()
pkg/util/metric/registry.go:265 +0x103
github.com/cockroachdb/cockroach/pkg/server/status.eachRecordableValue()
recorder.go:1074 +0xbb8
github.com/cockroachdb/cockroach/pkg/server/status.(*MetricsRecorder).GenerateNodeStatus()
recorder.go:913 +0xaa0
github.com/cockroachdb/cockroach/pkg/server.(*Node).writeNodeStatus.func1()
pkg/server/node.go:1576 +0x9d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTask()
pkg/util/stop/stopper.go:318 +0x236
github.com/cockroachdb/cockroach/pkg/server.(*Node).writeNodeStatus()
pkg/server/node.go:1575 +0x107
github.com/cockroachdb/cockroach/pkg/server.(*Node).startWriteNodeStatus.func1()
pkg/server/node.go:1538 +0x44
github.com/cockroachdb/cockroach/pkg/util/startup.RunIdempotentWithRetry.func1()
pkg/util/startup/retry.go:116 +0x3e
github.com/cockroachdb/cockroach/pkg/util/startup.RunIdempotentWithRetryEx[go.shape.interface {}]()
pkg/util/startup/retry.go:137 +0x289
github.com/cockroachdb/cockroach/pkg/util/startup.RunIdempotentWithRetry()
pkg/util/startup/retry.go:115 +0xa7
github.com/cockroachdb/cockroach/pkg/server.(*Node).startWriteNodeStatus()
pkg/server/node.go:1535 +0x12a
github.com/cockroachdb/cockroach/pkg/server.(*topLevelServer).PreStart()
pkg/server/server.go:2187 +0x5c30
github.com/cockroachdb/cockroach/pkg/server.(*topLevelServer).PreStart()
pkg/server/server.go:2063 +0x47c8
github.com/cockroachdb/cockroach/pkg/server.(*topLevelServer).PreStart()
pkg/server/server.go:1777 +0x23af
github.com/cockroachdb/cockroach/pkg/server.(*testServer).PreStart()
pkg/server/testserver.go:866 +0xfd
github.com/cockroachdb/cockroach/pkg/testutils/serverutils.(*wrap).PreStart()
bazel-out/k8-fastbuild/bin/pkg/testutils/serverutils/ts_control_forwarder_generated.go:19 +0x71
github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).Start()
pkg/testutils/testcluster/testcluster.go:527 +0x63a
github.com/cockroachdb/cockroach/pkg/testutils/serverutils.StartCluster()
pkg/testutils/serverutils/test_cluster_shim.go:341 +0x13d
pkg/ccl/telemetryccl/telemetryccl_test.TestBulkJobTelemetryLogging()
pkg/ccl/telemetryccl/telemetry_logging_test.go:228 +0x558
testing.tRunner()
GOROOT/src/testing/testing.go:1934 +0x21c
testing.(*T).Run.gowrap1()
GOROOT/src/testing/testing.go:1997 +0x44
==================
```
Parameters:
- attempt=1
- race=true
- run=2
- shard=2
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
/cc @cockroachlabs/obs-prs
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestBulkJobTelemetryLogging.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-67201
Contributor guide
Research direction
Reproduce pkg/ccl/telemetryccl/telemetry_logging_test.go:TestBulkJobTelemetryLogging with the reported race=true parameters and inspect the linked test report. Trace the failure through pkg/util/metric/registry.go:265, pkg/server/status/recorder.go, and the test's cluster startup at telemetry_logging_test.go:228. Done means the test passes reliably under the race configuration without the Registry.Each recursion failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100