matrixorigin / matrixorigin/matrixone
[Bug]: Data Branch MERGE returns success but a 100M-row branch remains different from its base
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
### Is there an existing issue for the same bug?
- [x] I have checked the existing issues.
### Branch Name
`4.2-dev` and `main`
### Commit ID
Confirmed examples:
- `4.2-dev`: `4066fffaf75b26d6836c1ebda01cdfe5aa22672e`
- `main`: `ab28d1b4ac41d0bef6b0800a2c6c455aaacd2fda`
### Other Environment Information
- TKE nightly Snapshot Backup/Restore workflow
- Test case: `cases/big_data_test/06_CLONE/nonsys_data_branch_100m.sql`
- Table: 100M rows with a primary key and secondary/unique indexes
### Actual Behavior
`DATA BRANCH MERGE` returns success after the branch updates 1000 rows, but the immediate post-merge `DATA BRANCH DIFF ... OUTPUT COUNT` returns `2000` instead of `0`.
Observed sequence in the 4.2-dev job:
```text
update ..._branch set col3 = col3 + 1 where id between 1 and 1000
cost: 4.729s
data branch merge ..._branch into ... when conflict accept
cost: 37.262s
(no SQL error)
data branch diff ..._branch against ... output count
Actual: 2000
Expected: 0
```
The merge reports success even though the source branch and destination table have not converged. `2000` is consistent with 1000 changed rows being represented as delete+insert differences, but the current evidence does not yet distinguish an apply failure from stale branch/LCA metadata.
This is repeatable across branches and dates:
- 4.2-dev: [Run 31278627669 / Job 93174211698](https://github.com/matrixorigin/mo-nightly-regression/actions/runs/31278627669/job/93174211698)
- 4.2-dev: [Run 31076332861 / Job 92578077800](https://github.com/matrixorigin/mo-nightly-regression/actions/runs/31076332861/job/92578077800)
- main: [Run 31054144872 / Job 92498736329](https://github.com/matrixorigin/mo-nightly-regression/actions/runs/31054144872/job/92498736329)
The same dedicated job has failed in every available Snapshot workflow run checked from 2026-07-20 through 2026-08-08. Therefore this is currently classified as a persistent correctness bug, not a newly introduced 4.2-dev regression.
### Expected Behavior
After a successful `data branch merge branch_table into base_table when conflict accept`, an immediate `data branch diff branch_table against base_table output count` must return `0`. If the merge cannot apply all changes, it must return an error rather than report success.
### Steps to Reproduce
```sql
data branch create table
big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
from
big_data_test_db_clone.table_with_pk_index_for_load_100M;
update big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
set col3 = col3 + 1
where id between 1 and 1000;
data branch diff
big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
against
big_data_test_db_clone.table_with_pk_index_for_load_100M
output count;
data branch merge
big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
into
big_data_test_db_clone.table_with_pk_index_for_load_100M
when conflict accept;
data branch diff
big_data_test_db_clone.table_with_pk_index_for_load_100M_branch
against
big_data_test_db_clone.table_with_pk_index_for_load_100M
output count;
```
Expected counts are `1000` before merge and `0` after merge. The nightly case returns `2000` after merge.
### Additional information
Relevant code path at the reported 4.2-dev commit:
1. `handleBranchMerge`
2. `diffMergeAgency`
3. `diffOnBase`
4. `mergeDiffs`
5. `sqlValuesAppender.flushAll`
6. `flushSqlValues`
- [`pkg/frontend/data_branch.go`](https://github.com/matrixorigin/matrixone/blob/4066fffaf75b26d6836c1ebda01cdfe5aa22672e/pkg/frontend/data_branch.go#L792-L1058)
- [`pkg/frontend/data_branch_output.go`](https://github.com/matrixorigin/matrixone/blob/4066fffaf75b26d6836c1ebda01cdfe5aa22672e/pkg/frontend/data_branch_output.go#L312-L390)
This is not a duplicate of [#24927](https://github.com/matrixorigin/matrixone/issues/24927): that issue was an explicit `formatValIntoString: not support type BLOB` failure. Its fix is present in the affected commits. The present defect is a successful MERGE followed by a non-zero diff.
Contributor guide
Assessment
This issue has not been assessed yet.