matrixorigin / matrixorigin/matrixone
[Bug] RENAME TABLE with multiple pairs is non-atomic; partial failures leave the database in inconsistent state
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Version
MatrixOne **v4.0.0-rc2** (`8.0.30-OmniFabric-v4.0.0-rc2`)
## Summary
`RENAME TABLE a TO b, c TO d, ...` does not behave atomically. If a later pair fails (or if the engine validates source/destination names upfront), the database is left in a half-renamed state with no rollback. Even splitting the swap into separate single-pair RENAME statements does not fully fix it — single-pair RENAMEs sometimes silently fail mid-loop and leave tables missing entirely.
This is sub-issue 2 of matrixone#25118 (atomic-swap ETL pattern broken).
## Reproduction
Classic 3-way swap idiom that works on MySQL 8 / MariaDB:
```sql
CREATE TABLE live (id INT);
CREATE TABLE shadow (id INT);
INSERT INTO shadow VALUES (1);
-- The standard atomic-swap pattern:
RENAME TABLE
live TO live_tmp,
shadow TO live,
live_tmp TO shadow;
```
Observed on MatrixOne:
```
ERROR 1050 (HY000): table 'live' already exists
```
The engine appears to validate the existence of every destination upfront before applying any pair, instead of processing pairs left-to-right. The middle pair (`shadow TO live`) sees `live` still in place because the first pair hasn't been applied yet.
### Splitting into separate statements still fails
Workaround we tried — three separate single-pair RENAMEs:
```sql
RENAME TABLE live TO live_tmp; -- step 1
RENAME TABLE shadow TO live; -- step 2
RENAME TABLE live_tmp TO shadow; -- step 3
```
In ~10% of iterations against our schema, step 3 silently fails. Outcome:
- `live` exists with the new data (step 2 succeeded)
- `shadow` does not exist anywhere (step 3 failed)
- `live_tmp` does not exist either
The original `shadow` table content has been **destroyed** without rollback. Subsequent iterations of the loop hit `ERROR 1146: no such table 'shadow'`.
We captured this exact failure on the loop log:
```
[2026-06-24 02:36:36] === Rebuild start ===
ERROR at line 1: ExpectedDup
[2026-06-24 02:36:43] silver=3s gold=0s swap=FAIL(gold_pump_kpis) live_gold_pump_kpis=8
[2026-06-24 02:41:43] === Rebuild start ===
ERROR 1146 (HY000) at line 1: no such table analytics.gold_pump_kpis_shadow
```
## Expected Behavior
MySQL contract:
- `RENAME TABLE a TO b, c TO d` is **atomic**: either all pairs apply or none do
- The whole statement holds a single global metadata lock until completion
- Pairs are processed left-to-right; later pairs see the state after earlier pairs were applied
- A failure in any pair **rolls back the entire statement**
When split into separate statements, each `RENAME TABLE` is its own atomic unit and must not destroy state silently.
## Actual Behavior
1. Multi-pair RENAME has upfront-validation semantics that reject the classic 3-way swap idiom
2. Single-pair RENAME occasionally fails silently mid-statement, destroying the source table without rollback
3. No rollback means recovery requires external bookkeeping (which is what the atomic-swap pattern was meant to *avoid*)
## Impact
- **The standard MySQL atomic-swap ETL pattern is unusable on MO** — confirmed in production-scale use case
- Users running blue/green table swaps, zero-downtime migrations, or any rebuild-then-promote pattern will lose data
- Half-swap state requires manual SQL recovery; loops cannot self-heal because of matrixone#25118 sub-issue 1 (CREATE TABLE IF NOT EXISTS is not silent)
## Suggested Fix
1. Implement true left-to-right processing for multi-pair `RENAME TABLE`, with rollback on any pair's failure (matches MySQL)
2. For single-pair `RENAME TABLE a TO b`: if `b` already exists OR if any internal error fires after step-1 metadata commit, **must roll back** before returning
3. Add a regression test: run 1000 iterations of the 3-way swap pattern and verify all tables stay populated
## Parent / Related
- Parent (use-case context): matrixone#25118
- Earlier RENAME multi-pair issue: matrixone#24408 (we filed in a previous version; can be closed as duplicate of this if the issue persists in v4.x)
Contributor guide
Assessment
This issue has not been assessed yet.