cockroachdb / cockroachdb/cockroach

pkg/ccl/benchccl/rttanalysisccl/rttanalysisccl_test: TestBenchmarkExpectation failed

Open
#173,378 0 comments 0 reactions 0 assignees View on GitHub
branch-release-25.4.15-rc C-test-failure O-robot release-blocker T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

pkg/ccl/benchccl/rttanalysisccl/rttanalysisccl_test.TestBenchmarkExpectation [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Ci_TestsAwsLinuxArm64_UnitTests/21466401?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Ci_TestsAwsLinuxArm64_UnitTests/21466401?buildTab=artifacts#/) on release-25.4.15-rc @ [675aa569da3140d32c117d49a9739b8f71368002](https://github.com/cockroachdb/cockroach/commits/675aa569da3140d32c117d49a9739b8f71368002):

```
=== RUN TestBenchmarkExpectation
I260814 20:49:50.872051 1 (gostd) rand.go:268 [-] 1 random seed: -2547010634221567481
test_log_scope.go:171: test logs captured to: /artifacts/tmp/_tmp/7418df5abfb13dbfaf6c0beee25f240a/logTestBenchmarkExpectation2356340564
test_log_scope.go:82: use -show-logs to present logs inline
validate_benchmark_data.go:120: success: expected AlterPrimaryRegion/alter_empty_database_set_initial_primary_region to perform KV lookups in [13, 13], got 13
validate_benchmark_data.go:120: success: expected AlterRegions/alter_empty_database_drop_region to perform KV lookups in [12, 16], got 14
validate_benchmark_data.go:120: success: expected AlterRegions/alter_empty_database_add_region to perform KV lookups in [11, 11], got 11
validate_benchmark_data.go:120: success: expected VirtualTableQueries/select_from_crdb_internal.zones_(10_tables) to perform KV lookups in [10, 10], got 10
validate_benchmark_data.go:120: success: expected AlterPrimaryRegion/alter_populated_database_alter_primary_region to perform KV lookups in [12, 14], got 12
validate_benchmark_data.go:120: success: expected AlterRegions/alter_populated_database_add_region to perform KV lookups in [11, 11], got 11
validate_benchmark_data.go:120: success: expected AlterRegions/alter_populated_database_drop_region to perform KV lookups in [14, 14], got 14
validate_benchmark_data.go:120: success: expected AlterPrimaryRegion/alter_empty_database_alter_primary_region to perform KV lookups in [12, 14], got 12
validate_benchmark_data.go:120: success: expected AlterSurvivalGoals/alter_empty_database_from_zone_to_region to perform KV lookups in [12, 12], got 12
validate_benchmark_data.go:120: success: expected AlterSurvivalGoals/alter_empty_database_from_region_to_zone to perform KV lookups in [12, 12], got 12
validate_benchmark_data.go:120: success: expected AlterSurvivalGoals/alter_populated_database_from_region_to_zone to perform KV lookups in [22, 22], got 22
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_regional_by_table_to_rbr to perform KV lookups in [18, 18], got 18
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_global_to_rbr to perform KV lookups in [18, 18], got 18
validate_benchmark_data.go:120: success: expected AlterPrimaryRegion/alter_populated_database_set_initial_primary_region to perform KV lookups in [14, 14], got 14
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_rbr_to_regional_by_table to perform KV lookups in [10, 10], got 10
validate_benchmark_data.go:120: success: expected VirtualTableQueries/select_from_crdb_internal.zones_(100_tables) to perform KV lookups in [10, 10], got 10
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_global_to_regional_by_table to perform KV lookups in [13, 13], got 13
validate_benchmark_data.go:120: success: expected AlterTableLocality/alter_from_regional_by_table_to_global to perform KV lookups in [13, 13], got 13
validate_benchmark_data.go:120: success: expected AlterSurvivalGoals/alter_populated_database_from_zone_to_region to perform KV lookups in [22, 22], got 22
validate_benchmark_data.go:120: success: expected VirtualTableQueries/select_from_crdb_internal.zones_(50_tables) to perform KV lookups in [10, 10], got 10
validate_benchmark_data.go:73: see the -rewrite flag to re-run the benchmarks and adjust the expectations
validate_benchmark_data.go:74: usage: ./dev test --rewrite pkg/bench/rttanalysis -f=TestBenchmarkExpectation/ --test-args '-rewrite-iterations=N'
validate_benchmark_data.go:107: -- test log scope end --
--- FAIL: TestBenchmarkExpectation (176.90s)
=== RUN TestBenchmarkExpectation/AlterTableLocality
--- FAIL: TestBenchmarkExpectation/AlterTableLocality (140.17s)
=== RUN TestBenchmarkExpectation/AlterTableLocality/alter_from_rbr_to_global
multi_region_bench_test.go:37: pq: setting updated but timed out waiting to read new value
--- FAIL: TestBenchmarkExpectation/AlterTableLocality/alter_from_rbr_to_global (61.91s)
```
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

/cc @cockroachlabs/sql-foundations

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestBenchmarkExpectation.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Contributor guide

Open the contributing guide

Research direction

Start with pkg/ccl/benchccl/rttanalysisccl/rttanalysisccl_test and the failure guidance in validate_benchmark_data.go. Run ./dev test --rewrite pkg/bench/rttanalysis -f=TestBenchmarkExpectation/ with the documented test arguments, then inspect multi_region_bench_test.go for the AlterTableLocality timeout. Done means TestBenchmarkExpectation passes reliably on the affected configuration without unexplained expectation changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.