dolthub / dolthub/doltlite

Conflict-resolution benchmark never executes conflict resolution

Closed
#2,893 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
268
Forks
18
Avg merge
2h 27m
Merged PRs (30d)
447

Description

The VC performance harness sends merge, resolution, inspection and rollback as one single-line SQL batch:

```sql
BEGIN; SELECT dolt_merge('feat'); SELECT dolt_conflicts_resolve('--ours','t'); SELECT count(*) FROM dolt_conflicts; ROLLBACK;
```

The expected merge-conflict error stops that batch before resolution. time_sql accepts any nonzero status when allow_error=1 and does not check a completion marker or postcondition, so the benchmark records a timing for an operation it never executed.

Reproduced at f5d4636e70 with one conflicting row: the benchmark-shaped script returns exit 1 and no stdout. Sending the statements on separate lines reaches resolution and a marker query prints AFTER_RESOLVE|0 (with exit 1 retained for the expected merge error). The conflict-count benchmark similarly never reaches its count query.

Execute operations separately, accept only the expected merge-conflict error, and require the expected conflict counts plus completion markers before accepting a timing sample. Add a harness regression proving that a skipped or broken resolver fails the benchmark.

Sources: https://github.com/dolthub/doltlite/blob/f5d4636e70/test/vc_perf_ceiling.sh#L441 and https://github.com/dolthub/doltlite/blob/f5d4636e70/test/vc_perf_ceiling.sh#L260

Contributor guide

No contributing guide indexed for this repository

Research direction

Open test/vc_perf_ceiling.sh at lines 260 and 441, then run the reproduced conflicting-row benchmark to observe the skipped statements and exit status. Separate the benchmark operations and verify the expected conflict counts and completion markers before accepting timing samples. Add a harness regression showing that a skipped or broken resolver fails the benchmark.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell, sql, sqlite
Domain
databases, performance, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.