AntidoteDB / AntidoteDB/antidote

Log efficiency

未关闭
#219 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
optimization
主要语言
Erlang
星标
892
派生
92
PR 合并指标
30 天内没有已合并 PR

描述

Currently the log is used for 3 things:
1. Loading the state of objects into the materialiser on restart (done by replaying the log).
2. Resending lost updates to external DCs.
3. Reading an older version of an object than what is available in memory.

Numbers 2. and 3. currently have major performance issues as they require traversing the entire log from the start.

Number 2. can be easily fixed by occasionally including disk_log continuations (basically pointers to locations in the log http://erlang.org/doc/man/disk_log.html#chunk-2) as part of the inter_dc_transactions. This would require little code change as processing the log is already done in chunks. (Note that 2. has another issue in that it sends all missing updates as a single binary blob, which might cause issues if DCs had been partitioned by for a long time, but this can also easily be fixed by sending pieces).

Number 3. would be more involved and probably need other techniques. Santiago is addressing this in his leveldb integration for multi versioning. He is also addressing the issue that can arise where there is not enough memory to keep the latest version of all objects in memory (currently Antidote assumes that the last few versions of all objects can fit in memory).

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。