External Write Ahead Log with relaxed guarantees
- Dominant language
- Go
- Stars
- 4
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
txfile ensures we have at least 1, and in the normal non corruption case 2 valid transactional states in the file. This is guaranteed by never overwriting any contents, but keeping an internal table with contents being changed. Drawback of this approach is, we always need 2 fsync operations to finish one transaction.
txfile by design guarantees only one write transaction. So to speed up writes/updates, we should provide optional support for writing to an external transaction log first. The syncing on the transaction log would be optional, and based on a flush timeout. All io will be append only until the point where we have to write the transaction log into the storage file (which could be done asynchronously if we keep the mapping tables for page updates). Drawback of postponing a flush/sync on the transaction log is a potential loss of transactions, still data will be consistent up the last known valid transaction state.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading txfile's transaction flow and current fsync behavior described in the issue. Define the external transaction-log interface, flush-timeout and recovery semantics, then verify that delayed or lost log syncing preserves consistency through the last valid transaction state.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100