mview: COMPLETE OUT OF PLACE refresh can start before CREATE MATERIALIZED VIEW finishes and only fail later at cutover
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
### 1. Minimal reproduce step (Required)
1. Create a materialized view on a base table large enough that `CREATE MATERIALIZED VIEW ... REFRESH FAST AS ...` stays in its initialization/build phase for a while.
2. Before the `CREATE MATERIALIZED VIEW` job finishes, run `REFRESH MATERIALIZED VIEW COMPLETE OUT OF PLACE`.
3. Let the out-of-place refresh finish the shadow-table build and enter cutover.
In one observed run on April 7, 2026:
- the `CREATE MATERIALIZED VIEW` job for the same MV did not finish until `2026-04-07 09:49:58 UTC`
- `REFRESH MATERIALIZED VIEW ... COMPLETE OUT OF PLACE` started at `2026-04-07 09:47:50 UTC`
- the refresh later failed at `2026-04-07 10:21:23 UTC`
Additional evidence from `mysql.tidb_ddl_history.job_meta` for that run:
- the create-MV job persisted `snapshot_ver = 465450650744651780`
- the refresh cutover job carried `expected_last_success_read_tso = 465450650731544588`
- the target MV's `mysql.tidb_mview_refresh_info.LAST_SUCCESS_READ_TSO` was already `465450650744651780` by the time cutover executed
This shows the refresh started while the create-MV initialization flow for the same MV was still mutating `LAST_SUCCESS_READ_TSO`.
### 2. What did you expect to see? (Required)
`REFRESH MATERIALIZED VIEW` should not run on a materialized view whose `CREATE MATERIALIZED VIEW` initialization has not finished yet.
Acceptable behavior would be one of:
- reject the refresh immediately with a clear state error
- or wait until create/initialization completes before starting the refresh
It should not spend a long time building a shadow table and only fail at the final cutover because initialization metadata changed underneath it.
### 3. What did you see instead (Required)
`REFRESH MATERIALIZED VIEW ... COMPLETE OUT OF PLACE` was accepted and ran for a long time, but finally failed at cutover with:
```text
[ddl:8204] Invalid DDL job ... refresh materialized view complete OUT OF PLACE cutover: stale LAST_SUCCESS_READ_TSO detected before cutover
```
This looks like a missing lifecycle/state guard rather than a result-correctness issue:
- the failure is safe
- the old MV remains serving
- but the error surfaces too late and exposes an internal metadata race instead of a user-facing "materialized view is still initializing" error
From code inspection, the behavior appears to come from this sequence:
- `CREATE MATERIALIZED VIEW` prewrites `mysql.tidb_mview_refresh_info.LAST_SUCCESS_READ_TSO` with a transaction start TSO and later overwrites it with the final create/build snapshot TSO
- `REFRESH MATERIALIZED VIEW ... COMPLETE OUT OF PLACE` reads `expected_last_success_read_tso` near the beginning and uses it as the cutover CAS expectation
- if refresh is allowed to start before create finishes, the later create-side metadata update can make the refresh cutover fail even though the conflicting writer is only the create job for the same MV
### 4. What is your TiDB version? (Required)
Observed on April 7, 2026 with:
```text
Release Version: v8.5.4
Edition: Enterprise
Git Commit Hash: 8907b97ad28ec3d14412a0a8d48de1d08e46baad
Git Branch: heads/refs/tags/v8.5.4
UTC Build Time: 2026-04-05 08:48:00
GoVersion: go1.23.6
Store: tikv
```
Contributor guide
Research direction
Trace the CREATE MATERIALIZED VIEW initialization and REFRESH MATERIALIZED VIEW COMPLETE OUT OF PLACE entry points, focusing on LAST_SUCCESS_READ_TSO and the cutover expectation. Reproduce the overlapping jobs, then verify that refresh is rejected or waits while initialization is active, rather than building a shadow table and failing at cutover.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, mysql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100