matrixorigin / matrixorigin/matrixone

[Bug] RENAME TABLE with multiple pairs is non-atomic; partial failures leave the database in inconsistent state

Open
#25,120 0 comments 0 reactions 1 assignee Claimed by @XuPeng-SH View on GitHub
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Version

MatrixOne **v4.0.0-rc2** (`8.0.30-OmniFabric-v4.0.0-rc2`)

## Summary

`RENAME TABLE a TO b, c TO d, ...` does not behave atomically. If a later pair fails (or if the engine validates source/destination names upfront), the database is left in a half-renamed state with no rollback. Even splitting the swap into separate single-pair RENAME statements does not fully fix it — single-pair RENAMEs sometimes silently fail mid-loop and leave tables missing entirely.

This is sub-issue 2 of matrixone#25118 (atomic-swap ETL pattern broken).

## Reproduction

Classic 3-way swap idiom that works on MySQL 8 / MariaDB:

```sql
CREATE TABLE live (id INT);
CREATE TABLE shadow (id INT);
INSERT INTO shadow VALUES (1);

-- The standard atomic-swap pattern:
RENAME TABLE
live TO live_tmp,
shadow TO live,
live_tmp TO shadow;
```

Observed on MatrixOne:
```
ERROR 1050 (HY000): table 'live' already exists
```

The engine appears to validate the existence of every destination upfront before applying any pair, instead of processing pairs left-to-right. The middle pair (`shadow TO live`) sees `live` still in place because the first pair hasn't been applied yet.

### Splitting into separate statements still fails

Workaround we tried — three separate single-pair RENAMEs:

```sql
RENAME TABLE live TO live_tmp; -- step 1
RENAME TABLE shadow TO live; -- step 2
RENAME TABLE live_tmp TO shadow; -- step 3
```

In ~10% of iterations against our schema, step 3 silently fails. Outcome:

- `live` exists with the new data (step 2 succeeded)
- `shadow` does not exist anywhere (step 3 failed)
- `live_tmp` does not exist either

The original `shadow` table content has been **destroyed** without rollback. Subsequent iterations of the loop hit `ERROR 1146: no such table 'shadow'`.

We captured this exact failure on the loop log:
```
[2026-06-24 02:36:36] === Rebuild start ===
ERROR at line 1: ExpectedDup
[2026-06-24 02:36:43] silver=3s gold=0s swap=FAIL(gold_pump_kpis) live_gold_pump_kpis=8

[2026-06-24 02:41:43] === Rebuild start ===
ERROR 1146 (HY000) at line 1: no such table analytics.gold_pump_kpis_shadow
```

## Expected Behavior

MySQL contract:
- `RENAME TABLE a TO b, c TO d` is **atomic**: either all pairs apply or none do
- The whole statement holds a single global metadata lock until completion
- Pairs are processed left-to-right; later pairs see the state after earlier pairs were applied
- A failure in any pair **rolls back the entire statement**

When split into separate statements, each `RENAME TABLE` is its own atomic unit and must not destroy state silently.

## Actual Behavior

1. Multi-pair RENAME has upfront-validation semantics that reject the classic 3-way swap idiom
2. Single-pair RENAME occasionally fails silently mid-statement, destroying the source table without rollback
3. No rollback means recovery requires external bookkeeping (which is what the atomic-swap pattern was meant to *avoid*)

## Impact

- **The standard MySQL atomic-swap ETL pattern is unusable on MO** — confirmed in production-scale use case
- Users running blue/green table swaps, zero-downtime migrations, or any rebuild-then-promote pattern will lose data
- Half-swap state requires manual SQL recovery; loops cannot self-heal because of matrixone#25118 sub-issue 1 (CREATE TABLE IF NOT EXISTS is not silent)

## Suggested Fix

1. Implement true left-to-right processing for multi-pair `RENAME TABLE`, with rollback on any pair's failure (matches MySQL)
2. For single-pair `RENAME TABLE a TO b`: if `b` already exists OR if any internal error fires after step-1 metadata commit, **must roll back** before returning
3. Add a regression test: run 1000 iterations of the 3-way swap pattern and verify all tables stay populated

## Parent / Related

- Parent (use-case context): matrixone#25118
- Earlier RENAME multi-pair issue: matrixone#24408 (we filed in a previous version; can be closed as duplicate of this if the issue persists in v4.x)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.