pingcap / pingcap/tidb

planner: cost fully engine-homogeneous plans that per-table engine selection never considers as a whole

Open
#70,021 0 comments 0 reactions 0 assignees View on GitHub
sig/planner type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

### Problem

During physical optimization the search picks a storage engine per `DataSource` through local, per-table cost comparisons. As a result it can settle on a plan whose engine choice is only locally optimal:

- It may **mix** TiKV and TiFlash reads without ever costing either fully engine-homogeneous plan as a whole.
- It may pick **all-TiKV** per-table index access even when a homogeneous whole-statement plan — in particular an MPP plan with the join/aggregation pushed to TiFlash — would be cheaper overall.

In both cases the homogeneous alternative is never costed as a single unit, so the optimizer cannot choose it even when it wins.

### Proposal

Add engine-restricted rounds to the alternative logical plan driver (behind `tidb_opt_enable_alternative_logical_plans`). After round 1, rebuild the plan with `IsolationReadEngines` restricted to a single storage engine (`tikv-only` / `tiflash-only`) and let the existing strict-`<` cross-round cost comparison pick the winner. Because a plan found under a narrower engine set is still valid under the user's wider set, the restricted-search winner can be used directly.

Guardrails so the extra optimization is only spent where it can pay off:

- Arm only for statements containing a join or aggregation (keeps the rounds off the OLTP fast path).
- Skip when a `READ_FROM_STORAGE` hint pins an engine (the cost comparison must not override an explicit user choice).
- Skip under enforced MPP (its cost discount would distort the cross-round comparison).
- Skip the tiflash-only round when any table has no TiFlash access path or when MPP is not allowed.

### Design alternatives under evaluation

Two prototypes explore *when* the rounds should arm; only one will ship.

- **#70005** arms the rounds only when round 1's chosen plan already mixes both engines (detected by walking its reader boundaries). Narrowest trigger.
- **#70020** arms each round from the engines round 1 actually read, so an all-TiKV plan still gets a whole-statement TiFlash/MPP alternative costed. Covers the all-TiKV case the first cannot reach, at the cost of arming more often.

This issue tracks the enhancement; both PRs `ref` it.

Contributor guide

Open the contributing guide

Research direction

Start in the alternative logical plan driver and trace how round 1 builds plans, applies IsolationReadEngines, and performs the strict-< cross-round cost comparison. Compare the approaches described in PRs #70005 and #70020, then verify that the selected design covers the stated join/aggregation, hint, enforced-MPP, TiFlash-access, and MPP guardrails.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.