cockroachdb / cockroachdb/cockroach

roachtest: clearrange/dense failed

Open
#169,634 7 comments 0 reactions 1 assignee Claimed by @sumeerbhola View on GitHub
A-storage branch-master C-test-failure O-roachtest O-robot T-storage
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

roachtest.clearrange/dense [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestWeeklyBazel/21335327?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestWeeklyBazel/21335327?buildTab=artifacts#/clearrange/dense) on master @ [5b0bc8ba6eb54d9e5080371c6dab17d465391d08](https://github.com/cockroachdb/cockroach/commits/5b0bc8ba6eb54d9e5080371c6dab17d465391d08):

Failed with:

```
(assertions.go:387).Fail:
Error Trace: pkg/cmd/roachtest/tests/clearrange.go:300
pkg/cmd/roachtest/tests/clearrange.go:149
pkg/cmd/roachtest/tests/clearrange.go:247
pkg/cmd/roachtest/monitor.go:233
external/org_golang_x_sync/errgroup/errgroup.go:93
src/runtime/asm_amd64.s:1771
Error: Received unexpected error:
pq: internal error while retrieving user account memberships: operation "get-user-session" timed out after 10s (given timeout 10s): internal error while retrieving user account: get auth info error: interrupted during singleflight acquire-lease:1: context deadline exceeded
Test: clearrange/dense
(require.go:1401).NoError: FailNow called
(cluster.go:2647).Run: context canceled
(monitor.go:267).Wait: monitor failure: monitor user task failed: t.Fatal() was called
test artifacts and logs in: /artifacts/clearrange/dense/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21335327-1777701051-102-n6cpu32-0001 | 34.26.149.14 | 10.142.0.160 |
| teamcity-21335327-1777701051-102-n6cpu32-0002 | 34.26.152.73 | 10.142.0.135 |
| teamcity-21335327-1777701051-102-n6cpu32-0003 | 34.139.244.17 | 10.142.0.186 |
| teamcity-21335327-1777701051-102-n6cpu32-0004 | 34.26.232.89 | 10.142.0.172 |
| teamcity-21335327-1777701051-102-n6cpu32-0005 | 34.75.152.45 | 10.142.0.148 |
| teamcity-21335327-1777701051-102-n6cpu32-0006 | 34.24.98.252 | 10.142.0.136 |

Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=32
- diskCount=0
- encrypted=false
- fs=ext4
- localSSD=false
- metamorphicDRPC=true
- metamorphicWriteBuffering=true
- runtimeAssertionsBuild=false
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21335327/clearrange-dense/1777748519723/1777794538556)

See: [Datadog Logs](https://us5.datadoghq.com/logs?query=service%3Aroachtest+%40cluster%3Ateamcity-21335327-1777701051-102-n6cpu32&from_ts=1777748519723&to_ts=1777794538556&live=false&storage=flex_tier)

/cc @cockroachdb/storage

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*clearrange/dense.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-63590

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.