cockroachdb / cockroachdb/cockroach
pkg/ccl/benchccl/rttanalysisccl/rttanalysisccl_test: TestBenchmarkExpectation failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
pkg/ccl/benchccl/rttanalysisccl/rttanalysisccl_test.TestBenchmarkExpectation [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Ci_TestsAwsLinuxArm64_UnitTests/21466401?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Ci_TestsAwsLinuxArm64_UnitTests/21466401?buildTab=artifacts#/) on release-25.4.15-rc @ [675aa569da3140d32c117d49a9739b8f71368002](https://github.com/cockroachdb/cockroach/commits/675aa569da3140d32c117d49a9739b8f71368002):
```
=== RUN TestBenchmarkExpectation
I260814 20:49:50.872051 1 (gostd) rand.go:268 [-] 1 random seed: -2547010634221567481
test_log_scope.go:171: test logs captured to: /artifacts/tmp/_tmp/7418df5abfb13dbfaf6c0beee25f240a/logTestBenchmarkExpectation2356340564
test_log_scope.go:82: use -show-logs to present logs inline
validate_benchmark_data.go:120: success: expected AlterPrimaryRegion/alter_empty_database_set_initial_primary_region to perform KV lookups in [13, 13], got 13
validate_benchmark_data.go:120: success: expected AlterRegions/alter_empty_database_drop_region to perform KV lookups in [12, 16], got 14
validate_benchmark_data.go:120: success: expected AlterRegions/alter_empty_database_add_region to perform KV lookups in [11, 11], got 11
validate_benchmark_data.go:120: success: expected VirtualTableQueries/select_from_crdb_internal.zones_(10_tables) to perform KV lookups in [10, 10], got 10
validate_benchmark_data.go:120: success: expected AlterPrimaryRegion/alter_populated_database_alter_primary_region to perform KV lookups in [12, 14], got 12
validate_benchmark_data.go:120: success: expected AlterRegions/alter_populated_database_add_region to perform KV lookups in [11, 11], got 11
validate_benchmark_data.go:120: success: expected AlterRegions/alter_populated_database_drop_region to perform KV lookups in [14, 14], got 14
validate_benchmark_data.go:120: success: expected AlterPrimaryRegion/alter_empty_database_alter_primary_region to perform KV lookups in [12, 14], got 12
validate_benchmark_data.go:120: success: expected AlterSurvivalGoals/alter_empty_database_from_zone_to_region to perform KV lookups in [12, 12], got 12
validate_benchmark_data.go:120: success: expected AlterSurvivalGoals/alter_empty_database_from_region_to_zone to perform KV lookups in [12, 12], got 12
validate_benchmark_data.go:120: success: expected AlterSurvivalGoals/alter_populated_database_from_region_to_zone to perform KV lookups in [22, 22], got 22
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_regional_by_table_to_rbr to perform KV lookups in [18, 18], got 18
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_global_to_rbr to perform KV lookups in [18, 18], got 18
validate_benchmark_data.go:120: success: expected AlterPrimaryRegion/alter_populated_database_set_initial_primary_region to perform KV lookups in [14, 14], got 14
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_rbr_to_regional_by_table to perform KV lookups in [10, 10], got 10
validate_benchmark_data.go:120: success: expected VirtualTableQueries/select_from_crdb_internal.zones_(100_tables) to perform KV lookups in [10, 10], got 10
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_global_to_regional_by_table to perform KV lookups in [13, 13], got 13
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_regional_by_table_to_global to perform KV lookups in [13, 13], got 13
validate_benchmark_data.go:120: success: expected AlterSurvivalGoals/alter_populated_database_from_zone_to_region to perform KV lookups in [22, 22], got 22
validate_benchmark_data.go:120: success: expected VirtualTableQueries/select_from_crdb_internal.zones_(50_tables) to perform KV lookups in [10, 10], got 10
validate_benchmark_data.go:73: see the -rewrite flag to re-run the benchmarks and adjust the expectations
validate_benchmark_data.go:74: usage: ./dev test --rewrite pkg/bench/rttanalysis -f=TestBenchmarkExpectation/ --test-args '-rewrite-iterations=N'
validate_benchmark_data.go:107: -- test log scope end --
--- FAIL: TestBenchmarkExpectation (176.90s)
=== RUN TestBenchmarkExpectation/AlterTableLocality
--- FAIL: TestBenchmarkExpectation/AlterTableLocality (140.17s)
=== RUN TestBenchmarkExpectation/AlterTableLocality/alter_from_rbr_to_global
multi_region_bench_test.go:37: pq: setting updated but timed out waiting to read new value
--- FAIL: TestBenchmarkExpectation/AlterTableLocality/alter_from_rbr_to_global (61.91s)
```
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
/cc @cockroachlabs/sql-foundations
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestBenchmarkExpectation.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Contributor guide
Research direction
Start with pkg/ccl/benchccl/rttanalysisccl/rttanalysisccl_test and the failure guidance in validate_benchmark_data.go. Run ./dev test --rewrite pkg/bench/rttanalysis -f=TestBenchmarkExpectation/ with the documented test arguments, then inspect multi_region_bench_test.go for the AlterTableLocality timeout. Done means TestBenchmarkExpectation passes reliably on the affected configuration without unexplained expectation changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100