pingcap / pingcap/tidb

expression: introduce unified short-circuit evaluation across TiDB and TiKV

Open
#70,156 0 comments 0 reactions 0 assignees View on GitHub
contribution type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

### Background

Some expressions do not need all arguments to be evaluated:

- `AND`: skip remaining arguments after `FALSE`.
- `OR`: skip remaining arguments after `TRUE`.
- `IF` / `CASE WHEN`: evaluate only the selected branch.
- `IFNULL` / `COALESCE`: stop after finding a non-`NULL` value.

TiDB's scalar `AND`/`OR` evaluator already short-circuits, but this behavior is not consistent across TiDB vectorized evaluation, TiKV pushdown, and other conditional functions.

Eager evaluation causes unnecessary computation and may produce warnings or errors from branches that should have been skipped. TiDB's vectorized evaluator may also fall back to scalar evaluation after a warning, resulting in repeated work.

### Benefits

A unified short-circuit framework can:

- Reduce CPU usage and query latency for expensive expressions.
- Avoid unnecessary warnings and errors.
- Avoid vectorized fallback and repeated evaluation.
- Keep TiDB root and TiKV pushdown behavior consistent.

### Proposed direction

Introduce a common lazy evaluation mechanism:

1. Evaluate the first argument for the current row set.
2. Determine which rows still require another argument.
3. Evaluate the next argument only for those rows.
4. Merge partial results while preserving SQL `TRUE`/`FALSE`/`NULL` semantics.

TiDB can implement this using `Chunk.Sel()` and result scattering. TiKV can introduce short-circuit expression nodes whose child expressions are evaluated on demand instead of eagerly through normal RPN evaluation.

Introduce a rollout switch:

```text
tidb_enable_short_circuit_expression
Scope: GLOBAL | SESSION
Type: BOOL
Default: OFF
```

TiDB should propagate the statement-level setting to TiKV through `DAGRequest.flags`.

### Work items

- [ ] Implement TiDB vectorized short-circuit evaluation for `AND` and `OR`.
- [ ] Implement TiKV short-circuit expression evaluation.
- [ ] Add the system variable and DAG request flag.
- [ ] Support short circuit for `IF`, `IFNULL`, `COALESCE`, and `CASE WHEN`.

Contributor guide

Open the contributing guide

Research direction

Start by tracing TiDB vectorized evaluation around Chunk.Sel() and statement flag propagation through DAGRequest.flags, then compare TiKV's normal RPN evaluation. Review the proposed system variable and the listed conditional expressions. Done means consistent short-circuit behavior across TiDB and TiKV, with SQL TRUE/FALSE/NULL semantics preserved and coverage for the rollout switch and all listed functions.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.