cockroachdb / cockroachdb/cockroach

roachtest: unoptimized-query-oracle/disable-rules=half/seed-multi-region failed

Open
#175,544 2 comments 0 reactions 0 assignees View on GitHub
B-runtime-assertions-enabled branch-master C-test-failure O-roachtest O-robot O-rsg release-blocker T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Note:** This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar failure without this message. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.

roachtest.unoptimized-query-oracle/disable-rules=half/seed-multi-region [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21518918?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21518918?buildTab=artifacts#/unoptimized-query-oracle/disable-rules=half/seed-multi-region) on master @ [17d4d0939905dacd9f19fb6a372bed8ac62a73fa](https://github.com/cockroachdb/cockroach/commits/17d4d0939905dacd9f19fb6a372bed8ac62a73fa):

Failed with:

```
(query_comparison_util.go:501).runOneRoundQueryComparison: internal error while running unoptimized statement. 243 statements run: pq: internal error: unhandled op distribute
test artifacts and logs in: /artifacts/unoptimized-query-oracle/disable-rules=half/seed-multi-region/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21518918-1789538360-61-n9cpu4-geo-0001 | | 10.91.98.186 |
| teamcity-21518918-1789538360-61-n9cpu4-geo-0002 | | 10.91.128.106 |
| teamcity-21518918-1789538360-61-n9cpu4-geo-0003 | | 10.91.48.116 |
| teamcity-21518918-1789538360-61-n9cpu4-geo-0004 | | 10.91.98.23 |
| teamcity-21518918-1789538360-61-n9cpu4-geo-0005 | | 10.91.128.32 |
| teamcity-21518918-1789538360-61-n9cpu4-geo-0006 | | 10.91.48.114 |
| teamcity-21518918-1789538360-61-n9cpu4-geo-0007 | | 10.91.97.72 |
| teamcity-21518918-1789538360-61-n9cpu4-geo-0008 | | 10.91.128.109 |
| teamcity-21518918-1789538360-61-n9cpu4-geo-0009 | | 10.91.48.115 |

Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=4
- diskCount=0
- encrypted=false
- metamorphicDRPC=true
- metamorphicWriteBuffering=true
- runtimeAssertionsBuild=true
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21518918/unoptimized-query-oracle-disable-rules-half-seed-multi-region/1789554256186/1789557658536)

See: [Datadog Logs](https://us5.datadoghq.com/logs?query=service%3Aroachtest+%40cluster%3Ateamcity-21518918-1789538360-61-n9cpu4-geo&from_ts=1789554256186&to_ts=1789557658536&live=false&storage=flex_tier)

Same failure on other branches

- #171476 roachtest: unoptimized-query-oracle/disable-rules=half/seed-multi-region failed [B-runtime-assertions-enabled C-bug O-roachtest O-robot O-rsg S-3-erroneous-edge-case T-sql-queries branch-release-24.3]

[Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-68451

Contributor guide

Open the contributing guide

Research direction

Start with the failure reported at query_comparison_util.go:501 and inspect the run_1 artifacts and logs for unoptimized-query-oracle/disable-rules=half/seed-multi-region. Read the roachtest README and compare the related failure in #171476; done means identifying and fixing the cause of the unhandled op distribute error, with the roachtest no longer failing.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.