cockroachdb / cockroachdb/cockroach
release-26.2.3-rc: roachtest: jepsen workload completed, but analysis timed out failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
roachtest.jepsen workload completed, but analysis timed out [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21404390?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21404390?buildTab=artifacts#/jepsen/g2/subcritical-skews) on release-26.2.3-rc @ [90d3b6080727dc810224a99903cb2e88b81e91ae](https://github.com/cockroachdb/cockroach/commits/90d3b6080727dc810224a99903cb2e88b81e91ae):
Failed with:
```
test jepsen/g2/subcritical-skews failed: (jepsen.go:458).runJepsen: TRANSIENT_ERROR(jepsen workload completed, but analysis timed out): timed out
test artifacts and logs in: /artifacts/jepsen/g2/subcritical-skews/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21404390-1782713363-105-n6cpu4-0001 | 34.74.73.152 | 10.142.3.8 |
| teamcity-21404390-1782713363-105-n6cpu4-0002 | 34.138.37.168 | 10.142.2.151 |
| teamcity-21404390-1782713363-105-n6cpu4-0003 | 104.196.14.231 | 10.142.2.158 |
| teamcity-21404390-1782713363-105-n6cpu4-0004 | 34.74.124.139 | 10.142.2.230 |
| teamcity-21404390-1782713363-105-n6cpu4-0005 | 34.148.110.189 | 10.142.2.224 |
| teamcity-21404390-1782713363-105-n6cpu4-0006 | 34.148.0.248 | 10.142.1.239 |
Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=4
- diskCount=0
- encrypted=false
- metamorphicLeases=default
- metamorphicWriteBuffering=true
- runtimeAssertionsBuild=false
Help
See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)
See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)
See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21404390/jepsen-g2-subcritical-skews/1782728156595/1782731094219)
/cc @cockroachlabs/test-eng
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*jepsen workload completed, but analysis timed out.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-65227
Contributor guide
Research direction
Start with the roachtest README and the jepsen/g2/subcritical-skews artifacts from TeamCity build 21404390. Review the failure logs and linked Grafana view to determine why analysis timed out after the workload completed. Done means the timeout cause is identified and the test or reporting behavior no longer produces this failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100