cockroachdb / cockroachdb/cockroach

roachtest: unoptimized-query-oracle/disable-rules=half/seed-multi-region failed

Open
#175,614 2 comments 0 reactions 0 assignees View on GitHub
branch-master C-test-failure O-roachtest O-robot O-rsg T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

roachtest.unoptimized-query-oracle/disable-rules=half/seed-multi-region [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21521842?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21521842?buildTab=artifacts#/unoptimized-query-oracle/disable-rules=half/seed-multi-region) on master @ [fc4bab5cd872ac0384f46264ff2cf147f26819cd](https://github.com/cockroachdb/cockroach/commits/fc4bab5cd872ac0384f46264ff2cf147f26819cd):

Failed with:

```
(query_comparison_util.go:501).runOneRoundQueryComparison: . 191 statements run: expected unoptimized and optimized results to be equal
[]string{
+ "NULL,-1363998557,-1363998557,0",
"NULL,-14,-14,0",
"NULL,-35,-35,0",
+ "NULL,-516271786,-516271786,0",
"NULL,1047798783,1047798783,0",
"NULL,22,22,0",
- "NULL,348252830512122019,348252830512122019,0",
- "NULL,7761279578671829334,7761279578671829334,0",
}
sql: SELECT
NULL AS col_774, tab_242.col_771 AS col_775, tab_242.col_771 AS col_776, tab_242.col_770 AS col_777
FROM
(
VALUES
(
0,
(
SELECT
tab_241._int4 AS col_769
FROM
defaultdb.public.seed AS tab_241
WHERE
('2b:a0:a0:2b:a0:2b:2b:a0/0'::INET >> tab_241._inet::INET)::BOOL
ORDER BY
col_769 DESC NULLS LAST
LIMIT
1
),
'010500004001000000010200004003000000706FD2A87DF6F1413D11CC1E2487FAC1301AC7D0A748024240960FBE4FD8B341F4518C754E4BFF4160AC94C1A1FCBF418089E8512D60AEC1240EA4C2A12B0242900AB7819D75FEC1',
87
),
(
0,
-35,
'010200008007000000BF3CBF66B967F8C16456417D8442F9C1600F1F8D2E4FC1C1A66E9DBBC167EEC1C0180B083C98E7C188C80FF459CED641FC2FD9C2BED1D5C1A0D414A5359EB0C1203F945C1DEBB6C180680657CF04B8411CA1384E1363D7C192148847677D0042942432C4578FF84118688BD2D019D94106EDF57BD5D3F041724B8A2D665EF241DE19CB5E813BF841F4D0604A91D0E8418281E57D8D79E5C1787A2F9F89840142A25CBFEBDB4FEBC1',
-8298577311956919160
),
(
0,
-14,
'0103000000010000000500000000000000000000000000000000000000000000000000F03F0000000000000000000000000000F03F000000000000F03F0000000000000000000000000000F03F00000000000000000000000000000000',
NULL
),
(0, 7761279578671829334, '0101000080000000000000F87F000000000000F87F000000000000F87F', 7295335700239008000),
(CASE WHEN true THEN 0 ELSE 0 END, 22, NULL, 2),
(
0,
348252830512122019,
'010200000002000000000000000000F03F000000000000F03F00000000000000400000000000000040',
127
)
)
AS tab_242 (col_770, col_771, col_772, col_773)
ORDER BY
col_775 NULLS FIRST,
col_776 ASC NULLS FIRST,
tab_242.col_773 NULLS FIRST,
col_774 NULLS FIRST,
col_777,
tab_242.col_772 ASC,
tab_242.col_771 ASC NULLS FIRST,
tab_242.col_770
LIMIT
18
test artifacts and logs in: /artifacts/unoptimized-query-oracle/disable-rules=half/seed-multi-region/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21521842-1789709875-65-n9cpu4-geo-0001 | | 10.91.99.51 |
| teamcity-21521842-1789709875-65-n9cpu4-geo-0002 | | 10.91.128.70 |
| teamcity-21521842-1789709875-65-n9cpu4-geo-0003 | | 10.91.48.66 |
| teamcity-21521842-1789709875-65-n9cpu4-geo-0004 | | 10.91.99.38 |
| teamcity-21521842-1789709875-65-n9cpu4-geo-0005 | | 10.91.128.69 |
| teamcity-21521842-1789709875-65-n9cpu4-geo-0006 | | 10.91.48.64 |
| teamcity-21521842-1789709875-65-n9cpu4-geo-0007 | | 10.91.99.104 |
| teamcity-21521842-1789709875-65-n9cpu4-geo-0008 | | 10.91.128.74 |
| teamcity-21521842-1789709875-65-n9cpu4-geo-0009 | | 10.91.48.65 |

Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=4
- diskCount=0
- encrypted=false
- metamorphicWriteBuffering=true
- runtimeAssertionsBuild=false
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21521842/unoptimized-query-oracle-disable-rules-half-seed-multi-region/1789728882396/1789732361922)

See: [Datadog Logs](https://us5.datadoghq.com/logs?query=service%3Aroachtest+%40cluster%3Ateamcity-21521842-1789709875-65-n9cpu4-geo&from_ts=1789728882396&to_ts=1789732361922&live=false&storage=flex_tier)

Same failure on other branches

- #171476 roachtest: unoptimized-query-oracle/disable-rules=half/seed-multi-region failed [B-runtime-assertions-enabled C-bug O-roachtest O-robot O-rsg S-3-erroneous-edge-case T-sql-queries branch-release-24.3]

[Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-68501

Contributor guide

Open the contributing guide

Research direction

Start at query_comparison_util.go:501 and read the roachtest README for the investigation workflow. Use the linked TeamCity artifacts and logs to investigate the unoptimized-query-oracle failure and compare the reported optimized and unoptimized results. Done means the seed-multi-region test no longer reports unequal results.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.