cockroachdb / cockroachdb/cockroach

basalt-26.3: cli: TestConcurrentZip failed

Open
#174,845 5 comments 0 reactions 0 assignees View on GitHub
branch-basalt-26.3 C-test-failure O-robot release-blocker T-observability
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

cli.TestConcurrentZip [failed](https://mesolite.cluster.engflow.com/invocations/default/eca4abfe-ac7d-40a6-b7e7-dff9e155ec34?testReportRun=2&testReportShard=3&testReportAttempt=1#targets-Ly9wa2cvY2xpOmNsaV90ZXN0) on basalt-26.3 @ [017b84cd0101b832a6ee0c1934d073d108d0b24a](https://github.com/cockroachdb/cockroach/commits/017b84cd0101b832a6ee0c1934d073d108d0b24a):

```
=== RUN TestConcurrentZip
test_log_scope.go:172: test logs captured to: outputs.zip/logTestConcurrentZip2220836449
test_log_scope.go:82: use -show-logs to present logs inline
test_log_scope.go:172: test logs captured to: outputs.zip/logTestConcurrentZip745236823
[debug --host=127.0.0.1:39757 --insecure=true zip --timeout=30s --cpu-profile-duration=0s --validate-zip-file=false --cluster-name=TestCluster-AKqUGmumTd /dev/null]
zip_test.go:472:
/mnt/engflow/worker/work/1/exec/bazel-out/k8-fastbuild/bin/pkg/cli/cli_test_/cli_test.runfiles/com_github_cockroachdb_cockroach/pkg/cli/testdata/zip/testzip_concurrent:1:
zip
output didn't match expected:
@@ -213,13 +213,13 @@
[node 3] requesting execution trace list: creating error output: debug/nodes/3/executiontraces.err.txt...
[node 3] requesting execution trace list: done
[node 3] requesting execution trace list: last request failed: rpc error: ...
[node 3] requesting execution trace list: received response...
[node 3] requesting goroutine dump list...
+[node 3] requesting goroutine dump list: WARNING: 1 log entries in clitest.dcd01dbd-e4d8-4285-a4df-11fbaebf493a.roach.2026-09-08T08_05_56Z.000054.log mentioned sensitive cluster settings and were scrubbed client-side (0 were not redactable and were replaced entirely)
[node 3] requesting goroutine dump list: creating error output: debug/nodes/3/goroutines.err.txt...
[node 3] requesting goroutine dump list: done
-[node 3] requesting goroutine dump list: last request failed: rpc error: ...
[node 3] requesting goroutine dump list: received response...
[node 3] requesting goroutine profile...
[node 3] requesting goroutine profile: done
[node 3] requesting goroutine profile: received response...
[node 3] requesting goroutine profile: writing binary output: debug/nodes/3/stacks.pprof...
@@ -252,6 +252,7 @@
[node 3] writing ranges: writing JSON output: debug/nodes/3/ranges.json...
[node ?] ? log files found
[node ?] ? log files found
[node ?] ? log files found
debug zip --timeout=30s --cpu-profile-duration=0s --validate-zip-file=false --cluster-name= /dev/null
+last request failed: rpc error: ...

panic.go:694: -- test log scope end --
panic.go:694: -- test log scope end --
test logs left over in: outputs.zip/logTestConcurrentZip745236823
--- FAIL: TestConcurrentZip (26.77s)
```

Parameters:
- attempt=1
- deadlock=true
- run=2
- shard=3
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

Same failure on other branches

- #174844 release-26.2: cli: TestConcurrentZip failed [C-test-failure O-robot T-observability branch-release-26.2 release-blocker]
- #174810 cli: TestConcurrentZip failed [C-test-failure O-robot T-db-server T-observability branch-release-25.2.24-rc release-blocker]

/cc @cockroachlabs/obs-prs @cockroachlabs/server @cockroachlabs/obs-india-prs

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestConcurrentZip.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-68035

Contributor guide

Open the contributing guide

Research direction

Start with pkg/cli/zip_test.go around line 472 and the golden testdata at pkg/cli/testdata/zip/testzip_concurrent. Re-run TestConcurrentZip with the linked failure logs and compare the concurrent output, including the sensitive-settings warning and final RPC error. Done means the failure cause is understood and the test passes reliably without unexplained output differences.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.