matrixorigin / matrixorigin/matrixone
bug(lineage-gc): defer expected SI write conflicts as contention
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Description
`data_branch_lineage_gc` validates its fixed-SI discovery by writing the shared lineage-owner lifecycle row before commit. If an owner publisher commits after discovery, the validation transaction correctly loses with `ErrTxnWWConflict`. However, `isDataBranchLineageGCContention` does not classify `ErrTxnWWConflict` as expected contention, so `dataBranchLineageGCExecutorWithBudget` returns an error instead of safely deferring the maintenance run.
Safety is preserved because the stale GC transaction rolls back. The bug is operational/liveness behavior: normal owner-vs-GC races are reported as task failures and can cause recurring error logs, failed task metrics, and scheduler retries during snapshot/PITR/restore/DDL activity.
## Reproduction
Validated on `10.222.1.55` using NVMe storage and exact main head `d57753d442c5c19d9c490c4c393fba80f1e717f5` (which contains #27826).
A deterministic real two-CN embedded test performs:
1. CN1 starts an explicit SI transaction and reads active PITRs, fixing the GC discovery snapshot.
2. CN0 creates and commits a cluster PITR, crossing the lifecycle row in the normal frontend path.
3. CN1 writes `LineageOwnerLifecycleLockSQL()` with `WaitPolicy_FastFail` and commits.
Five consecutive runs produced the expected real storage error:
```text
update mo_catalog.mo_feature_registry ... feature_code = SNAPSHOT
...
tn.handle.commit.error error="w-w conflict: tableID: 0"
```
The returned error is `moerr.ErrTxnWWConflict` (`20619`). This is the exact stale-discovery validation race described in `compactExpiredAlterDataBranchLineageBatchWithExecutor`.
## Root cause
`pkg/sql/compile/alter_lineage_gc.go:isDataBranchLineageGCContention` includes:
- `ErrLockConflict`
- `ErrLockWaitTimeout`
- `ErrTxnNeedRetry`
- `ErrTxnNeedRetryWithDefChanged`
but omits `ErrTxnWWConflict`. The focused regression `TestDataBranchLineageGCExecutorDefersAfterContentionRollback` mirrors the same incomplete list, so mock coverage did not exercise the real SI commit result.
## Expected behavior
Treat `ErrTxnWWConflict` from this private SI GC work unit as expected contention after first giving parent cancellation precedence, exactly like the existing retry/lock conflict cases. The transaction must remain rolled back and the next scheduled invocation must rediscover from a fresh snapshot.
Add both:
- a focused executor regression using `moerr.NewTxnWWConflictNoCtx(...)`; and
- a real two-CN phase-controlled test where PITR publication lands between discovery and lifecycle validation.
## Regression source
Introduced by the GC contention classification in #27826 (`f91ce0647f5e1d8d9dcfb38a4d87063c739f074c`).
## Relationship to #27831
This condition was introduced by #27826 and remains independent of #27831's monotonic PITR-retention work. #27831 may exercise the same lineage-GC scheduler area, but it does not change this contention classification; the issue is therefore non-blocking for #27831 and must be fixed with its own paired contention regression.
Contributor guide
Assessment
This issue has not been assessed yet.