pingcap / pingcap/tidb

Support setting per-table ANALYZE options in CREATE TABLE and ALTER TABLE

Open
#70,962 0 comments 0 reactions 0 assignees View on GitHub
component/statistics sig/planner type/feature-request
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Feature Request

**Is your feature request related to a problem? Please describe:**

Today the only way to persist an analyze option (buckets, topn, sample num, sample rate,
column choice/list) is to run `ANALYZE TABLE t WITH ...`, which also collects statistics.
That has several consequences:

- Changing one option on a large table costs a full statistics collection nobody asked for.
- Options cannot be declared up front in a schema migration, before the table has data.
- Options are invisible in `SHOW CREATE TABLE`, so schema diffing and code review cannot see them.
- Options live in `mysql.analyze_options` keyed by physical table id, so they are silently lost on
`TRUNCATE TABLE`, on `EXCHANGE PARTITION`, and on BR restore. BR lists `analyze_options` in
`unRecoverableTable` (br/pkg/restore/snap_client/systable_restore.go) precisely because the rows
are keyed by table id.

pingcap/tidb#69853 added `ANALYZE ... WITH DEFAULT ` to clear a persisted option, which
makes the missing other half more visible: you can now reset an option without choosing a value,
but you still cannot set one without collecting statistics.

**Describe the feature you'd like:**

CREATE TABLE t (a int) STATS_BUCKETS = 64, STATS_TOPN = 20;
ALTER TABLE t STATS_SAMPLE_RATE = 0.5;
ALTER TABLE t STATS_BUCKETS = DEFAULT; -- clear, same reset spelling as #69853

No statistics are collected by these statements. The next ANALYZE, manual or auto, picks the
options up.

**Existing half-built scaffolding in the tree (important):**

This is as much "finish or delete dead code" as it is a new feature. Already present and unused:

- Grammar `ALTER TABLE t STATS_OPTIONS = 'str' | DEFAULT` (pkg/parser/parser.y:2283)
- AST `ast.StatsOptionsSpec` and `ast.AlterTableStatsOptions` (pkg/parser/ast/ddl.go:5830, :4056)
- Metadata `model.TableInfo.StatsOptions *model.StatsOptions` (pkg/meta/model/table.go:210) holding
AutoRecalc, ColumnChoice, ColumnList, SampleNum, SampleRate, Buckets, TopN, Concurrency and an
embedded `*StatsWindowSettings`
- DDL action `model.ActionAlterTableStatsOptions = 58` (pkg/meta/model/job.go:102), referenced only
by BDR role tests

There is no executor: `ast.AlterTableStatsOptions` is absent from the ALTER TABLE switch in
pkg/ddl/executor.go, so the statement falls through to `default: ErrUnsupportedAlterTableSpec`, and
nothing anywhere reads `TableInfo.StatsOptions`. Because the JSON field has existed for years and
was never written, reviving it needs no TableInfo version bump and old binaries simply ignore it.

Note also that MySQL's `STATS_PERSISTENT`, `STATS_AUTO_RECALC` and `STATS_SAMPLE_PAGES` table
options are already parsed and ignored by TiDB (pkg/parser/misc.go:808-819), so any new option
names must not collide with them and should not be confused with them in documentation.

**Describe alternatives you've considered:**

1. Keep `mysql.analyze_options` as the single store and have CREATE/ALTER TABLE write rows into it.
Smallest change and one source of truth, but the write is not atomic with the schema change, is
invisible to `SHOW CREATE TABLE`, and keeps the truncate / exchange partition / BR data loss.
2. Store in `TableInfo`, finishing the scaffolding above. Survives truncate, exchange partition and
BR restore, appears in `SHOW CREATE TABLE`, versioned like any other schema change. Cost: two
stores, so an explicit precedence rule against `mysql.analyze_options` is required.
3. A new system table keyed by schema and table name instead of id. Fixes truncate and BR without
DDL work, but invents a third stats configuration store and still misses `SHOW CREATE TABLE`.

**Teachability, Documentation, Adoption, Migration Strategy:**

Proposed precedence, most specific first:

statement WITH options
> options persisted by a previous ANALYZE ... WITH (mysql.analyze_options)
> options declared in DDL (TableInfo)
> tidb_analyze_default_num_buckets / _num_topn and the other system defaults

`DEFAULT` is the reset spelling at every level, consistent with #69853.

Scope: table level first. Per-index and per-column options do not exist anywhere today, since
`mysql.analyze_options` is keyed by physical table id and only `column_choice` / `column_ids` come
close, so they need their own design and should be a follow-up.

**Open questions for discussion:**

1. Store: `TableInfo` (alternative 2) or `mysql.analyze_options` (alternative 1)?
2. Syntax: first-class options (`STATS_BUCKETS`, `STATS_TOPN`, ...) or the existing
`STATS_OPTIONS='buckets=64,topn=20'` string blob already in the grammar?
3. Does `tidb_persist_analyze_options = OFF` suppress DDL-declared options, or are they schema and
therefore always in effect?
4. Partition level: is `ALTER TABLE t PARTITION p STATS_...` in scope? This interacts with #69952.
5. What happens to the unused `StatsWindowSettings` and `Concurrency` fields on `model.StatsOptions`:
keep unset, or strip them in the same change?

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing grammar in pkg/parser/parser.y, the AST types in pkg/parser/ast/ddl.go, and the ALTER TABLE dispatch in pkg/ddl/executor.go. Trace model.TableInfo.StatsOptions in pkg/meta/model/table.go and ActionAlterTableStatsOptions in pkg/meta/model/job.go, then resolve the open questions about syntax, storage, precedence, and partition scope. Done means the chosen design is implemented for table-level CREATE and ALTER TABLE options, with DEFAULT clearing values and no statistics collection.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.