elastic / elastic/go-txfile

External Write Ahead Log with relaxed guarantees

Open
#25 0 comments 0 reactions 0 assignees View on GitHub
[zube]: Backlog enhancement
Dominant language
Go
Stars
4
Forks
7
PR merge metrics
No merged PRs in 30d

Description

txfile ensures we have at least 1, and in the normal non corruption case 2 valid transactional states in the file. This is guaranteed by never overwriting any contents, but keeping an internal table with contents being changed. Drawback of this approach is, we always need 2 fsync operations to finish one transaction.

txfile by design guarantees only one write transaction. So to speed up writes/updates, we should provide optional support for writing to an external transaction log first. The syncing on the transaction log would be optional, and based on a flush timeout. All io will be append only until the point where we have to write the transaction log into the storage file (which could be done asynchronously if we keep the mapping tables for page updates). Drawback of postponing a flush/sync on the transaction log is a potential loss of transactions, still data will be consistent up the last known valid transaction state.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading txfile's transaction flow and current fsync behavior described in the issue. Define the external transaction-log interface, flush-timeout and recovery semantics, then verify that delayed or lost log syncing preserves consistency through the last valid transaction state.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.