matrixorigin / matrixorigin/matrixone

[Bug]: Data Branch MERGE returns success but a 100M-row branch remains different from its base

Open
#26,819 4 comments 0 reactions 1 assignee Claimed by @gouhongshen View on GitHub
ai-easy deferred kind/bug
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

### Is there an existing issue for the same bug?

- [x] I have checked the existing issues.

### Branch Name

`4.2-dev` and `main`

### Commit ID

Confirmed examples:

- `4.2-dev`: `4066fffaf75b26d6836c1ebda01cdfe5aa22672e`
- `main`: `ab28d1b4ac41d0bef6b0800a2c6c455aaacd2fda`

### Other Environment Information

- TKE nightly Snapshot Backup/Restore workflow
- Test case: `cases/big_data_test/06_CLONE/nonsys_data_branch_100m.sql`
- Table: 100M rows with a primary key and secondary/unique indexes

### Actual Behavior

`DATA BRANCH MERGE` returns success after the branch updates 1000 rows, but the immediate post-merge `DATA BRANCH DIFF ... OUTPUT COUNT` returns `2000` instead of `0`.

Observed sequence in the 4.2-dev job:

```text
update ..._branch set col3 = col3 + 1 where id between 1 and 1000
cost: 4.729s

data branch merge ..._branch into ... when conflict accept
cost: 37.262s
(no SQL error)

data branch diff ..._branch against ... output count
Actual: 2000
Expected: 0
```

The merge reports success even though the source branch and destination table have not converged. `2000` is consistent with 1000 changed rows being represented as delete+insert differences, but the current evidence does not yet distinguish an apply failure from stale branch/LCA metadata.

This is repeatable across branches and dates:

- 4.2-dev: [Run 31278627669 / Job 93174211698](https://github.com/matrixorigin/mo-nightly-regression/actions/runs/31278627669/job/93174211698)
- 4.2-dev: [Run 31076332861 / Job 92578077800](https://github.com/matrixorigin/mo-nightly-regression/actions/runs/31076332861/job/92578077800)
- main: [Run 31054144872 / Job 92498736329](https://github.com/matrixorigin/mo-nightly-regression/actions/runs/31054144872/job/92498736329)

The same dedicated job has failed in every available Snapshot workflow run checked from 2026-07-20 through 2026-08-08. Therefore this is currently classified as a persistent correctness bug, not a newly introduced 4.2-dev regression.

### Expected Behavior

After a successful `data branch merge branch_table into base_table when conflict accept`, an immediate `data branch diff branch_table against base_table output count` must return `0`. If the merge cannot apply all changes, it must return an error rather than report success.

### Steps to Reproduce

```sql
data branch create table
big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
from
big_data_test_db_clone.table_with_pk_index_for_load_100M;

update big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
set col3 = col3 + 1
where id between 1 and 1000;

data branch diff
big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
against
big_data_test_db_clone.table_with_pk_index_for_load_100M
output count;

data branch merge
big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
into
big_data_test_db_clone.table_with_pk_index_for_load_100M
when conflict accept;

data branch diff
big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
against
big_data_test_db_clone.table_with_pk_index_for_load_100M
output count;
```

Expected counts are `1000` before merge and `0` after merge. The nightly case returns `2000` after merge.

### Additional information

Relevant code path at the reported 4.2-dev commit:

1. `handleBranchMerge`
2. `diffMergeAgency`
3. `diffOnBase`
4. `mergeDiffs`
5. `sqlValuesAppender.flushAll`
6. `flushSqlValues`

- [`pkg/frontend/data_branch.go`](https://github.com/matrixorigin/matrixone/blob/4066fffaf75b26d6836c1ebda01cdfe5aa22672e/pkg/frontend/data_branch.go#L792-L1058)
- [`pkg/frontend/data_branch_output.go`](https://github.com/matrixorigin/matrixone/blob/4066fffaf75b26d6836c1ebda01cdfe5aa22672e/pkg/frontend/data_branch_output.go#L312-L390)

This is not a duplicate of [#24927](https://github.com/matrixorigin/matrixone/issues/24927): that issue was an explicit `formatValIntoString: not support type BLOB` failure. Its fix is present in the affected commits. The present defect is a successful MERGE followed by a non-zero diff.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.