cockroachdb / cockroachdb/cockroach

sql: investigate "steering" mutations in randomized tests to increase variety of query plans

Open
#98,292 0 comments 0 reactions 0 assignees View on GitHub
C-enhancement T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Is your feature request related to a problem? Please describe.**
We recently discovered a bug in our query engine, https://github.com/cockroachdb/cockroach/issues/97090, which could cause some queries planned with zigzag joins to produce incorrect results. This happened in the rare situation when the two indexes used for the zigzag join had equality columns with different directions (i.e., one `ASC` and one `DESC`), and multiple rows matched the query predicate. As part of the postmortem, we identified that part of the reason we didn't find this issue earlier was that our randomized tests never produced this scenario.

**Describe the solution you'd like**
A recent paper [1] describes a method for ensuring that a wider variety of plans are tested during randomized testing by "steering" the plans produced with well-timed and carefully chosen mutations to the database state (e.g., `CREATE INDEX`, `SET CLUSTER SETTING`, etc.). We should investigate whether we can do something similar with our randomized correctness tests.

[1] Jinsheng Ba and Manuel Rigger. "[Testing Database Engines via Query Plan Guidance](http://jinshengba.me/assets/pdf/qpg_icse23.pdf)" Proceedings of the 45th International Conference on Software Engineering. ICSE (2023).

Jira issue: CRDB-25172

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.