Conflict-resolution benchmark never executes conflict resolution
- Dominant language
- C
- Stars
- 268
- Forks
- 18
- Avg merge
- 2h 27m
- Merged PRs (30d)
- 447
Description
The VC performance harness sends merge, resolution, inspection and rollback as one single-line SQL batch:
```sql
BEGIN; SELECT dolt_merge('feat'); SELECT dolt_conflicts_resolve('--ours','t'); SELECT count(*) FROM dolt_conflicts; ROLLBACK;
```
The expected merge-conflict error stops that batch before resolution. time_sql accepts any nonzero status when allow_error=1 and does not check a completion marker or postcondition, so the benchmark records a timing for an operation it never executed.
Reproduced at f5d4636e70 with one conflicting row: the benchmark-shaped script returns exit 1 and no stdout. Sending the statements on separate lines reaches resolution and a marker query prints AFTER_RESOLVE|0 (with exit 1 retained for the expected merge error). The conflict-count benchmark similarly never reaches its count query.
Execute operations separately, accept only the expected merge-conflict error, and require the expected conflict counts plus completion markers before accepting a timing sample. Add a harness regression proving that a skipped or broken resolver fails the benchmark.
Sources: https://github.com/dolthub/doltlite/blob/f5d4636e70/test/vc_perf_ceiling.sh#L441 and https://github.com/dolthub/doltlite/blob/f5d4636e70/test/vc_perf_ceiling.sh#L260
Contributor guide
No contributing guide indexed for this repository
Research direction
Open test/vc_perf_ceiling.sh at lines 260 and 441, then run the reproduced conflicting-row benchmark to observe the skipped statements and exit status. Separate the benchmark operations and verify the expected conflict counts and completion markers before accepting timing samples. Add a harness regression showing that a skipped or broken resolver fails the benchmark.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell, sql, sqlite
- Domain
- databases, performance, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100