pingcap / pingcap/tidb

mview: COMPLETE OUT OF PLACE refresh can start before CREATE MATERIALIZED VIEW finishes and only fail later at cutover

Open
#67,594 0 comments 0 reactions 0 assignees View on GitHub
component/mview feature/developing severity/moderate type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

1. Create a materialized view on a base table large enough that `CREATE MATERIALIZED VIEW ... REFRESH FAST AS ...` stays in its initialization/build phase for a while.
2. Before the `CREATE MATERIALIZED VIEW` job finishes, run `REFRESH MATERIALIZED VIEW COMPLETE OUT OF PLACE`.
3. Let the out-of-place refresh finish the shadow-table build and enter cutover.

In one observed run on April 7, 2026:

- the `CREATE MATERIALIZED VIEW` job for the same MV did not finish until `2026-04-07 09:49:58 UTC`
- `REFRESH MATERIALIZED VIEW ... COMPLETE OUT OF PLACE` started at `2026-04-07 09:47:50 UTC`
- the refresh later failed at `2026-04-07 10:21:23 UTC`

Additional evidence from `mysql.tidb_ddl_history.job_meta` for that run:

- the create-MV job persisted `snapshot_ver = 465450650744651780`
- the refresh cutover job carried `expected_last_success_read_tso = 465450650731544588`
- the target MV's `mysql.tidb_mview_refresh_info.LAST_SUCCESS_READ_TSO` was already `465450650744651780` by the time cutover executed

This shows the refresh started while the create-MV initialization flow for the same MV was still mutating `LAST_SUCCESS_READ_TSO`.

### 2. What did you expect to see? (Required)

`REFRESH MATERIALIZED VIEW` should not run on a materialized view whose `CREATE MATERIALIZED VIEW` initialization has not finished yet.

Acceptable behavior would be one of:

- reject the refresh immediately with a clear state error
- or wait until create/initialization completes before starting the refresh

It should not spend a long time building a shadow table and only fail at the final cutover because initialization metadata changed underneath it.

### 3. What did you see instead (Required)

`REFRESH MATERIALIZED VIEW ... COMPLETE OUT OF PLACE` was accepted and ran for a long time, but finally failed at cutover with:

```text
[ddl:8204] Invalid DDL job ... refresh materialized view complete OUT OF PLACE cutover: stale LAST_SUCCESS_READ_TSO detected before cutover
```

This looks like a missing lifecycle/state guard rather than a result-correctness issue:

- the failure is safe
- the old MV remains serving
- but the error surfaces too late and exposes an internal metadata race instead of a user-facing "materialized view is still initializing" error

From code inspection, the behavior appears to come from this sequence:

- `CREATE MATERIALIZED VIEW` prewrites `mysql.tidb_mview_refresh_info.LAST_SUCCESS_READ_TSO` with a transaction start TSO and later overwrites it with the final create/build snapshot TSO
- `REFRESH MATERIALIZED VIEW ... COMPLETE OUT OF PLACE` reads `expected_last_success_read_tso` near the beginning and uses it as the cutover CAS expectation
- if refresh is allowed to start before create finishes, the later create-side metadata update can make the refresh cutover fail even though the conflicting writer is only the create job for the same MV

### 4. What is your TiDB version? (Required)

Observed on April 7, 2026 with:

```text
Release Version: v8.5.4
Edition: Enterprise
Git Commit Hash: 8907b97ad28ec3d14412a0a8d48de1d08e46baad
Git Branch: heads/refs/tags/v8.5.4
UTC Build Time: 2026-04-05 08:48:00
GoVersion: go1.23.6
Store: tikv
```

Contributor guide

Open the contributing guide

Research direction

Trace the CREATE MATERIALIZED VIEW initialization and REFRESH MATERIALIZED VIEW COMPLETE OUT OF PLACE entry points, focusing on LAST_SUCCESS_READ_TSO and the cutover expectation. Reproduce the overlapping jobs, then verify that refresh is rejected or waits while initialization is active, rather than building a shadow table and failing at cutover.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, mysql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.