matrixorigin / matrixorigin/matrixone

[Feature Request]: Support schema evolution in DATA BRANCH DIFF / MERGE

Open
#24,549 2 comments 0 reactions 1 assignee Claimed by @Ariznawlll View on GitHub
phase/testing
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

**Version:** `8.0.30-MatrixOne-v3.0.11` (MatrixOne v3.0.11)

`DATA BRANCH DIFF/MERGE` require the two tables to have an **identical schema**. Adding a column on a branch (a normal data-evolution step, e.g. a new feature column) breaks diff/merge.

**Reproduce**
```sql
CREATE TABLE a(id INT PRIMARY KEY, f0 DOUBLE);
DATA BRANCH CREATE TABLE b FROM a;
ALTER TABLE b ADD COLUMN f1 DOUBLE DEFAULT 0.0;
DATA BRANCH DIFF b AGAINST a;
```

**Actual**
```
ERROR 20101 (HY000): internal error: the target table schema is not equivalent to the base table.
```

**Request**
Support diff/merge across compatible schema changes (added / nullable columns), which is essential for evolving feature/label tables across versions.

---
_Found while evaluating MatrixOne git4data / SQL as a data-versioning & agent-trace backend. Version: `8.0.30-MatrixOne-v3.0.11` (MatrixOne v3.0.11)._

## Testing background
Found in a **feature-store experiment**: a feature definition changed on a branch (we added a feature column via `ALTER TABLE ... ADD COLUMN`), then we tried to `DATA BRANCH DIFF`/`MERGE` the new feature values against the base.

## Why it matters (expected purpose)
Feature/label/training tables **evolve over time** (new features get added). Requiring identical schemas means any column addition breaks diff/merge, blocking continuous feature engineering on branches. Supporting compatible schema evolution (added / nullable columns) is needed for iterative dataset work.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.