dolthub / dolthub/dolt

Implement Lazy Flush

Open
#11,084 0 comments 0 reactions 1 assignee Claimed by @jycor View on GitHub
performance
Dominant language
Go
Stars
24.4k
Forks
873
Avg merge
1d 5h
Merged PRs (30d)
108

Description

It's possible to gain write performance (in TPCC) through lazy flush.

Currently, dolt will always flush writes to tables. This means that INSERTs, UPDATEs, and DELETEs to `prollyTableWriter` are immediately materialized to the working set in `prollyWriteSession`. Alternatively, we should accumulate these edits and materialize them all at once.

However, we need to ensure that following queries read the most recent version of that table.
It's not enough to hold off on flushing until a `SELECT`, `ALTER`, etc because even UPDATEs need the newest version.
For example:
```
autocommit = 0;
create table t (i int);
insert into t values (0);
update t set i = i + 1;
```

I believe the right approach would be to have reads and writes be able to detect "dirty" write session and read rows from the mutable maps (instead of looking at the current working set).

Previous attempts with benchmarks for reference:
https://github.com/dolthub/dolt/pull/11048
https://github.com/dolthub/dolt/pull/11009

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.