Agent-Hellboy / Agent-Hellboy/hcp

In-Memory LSM Tree (Memtable) Support in Clipboard Daemon

未关闭
#10 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C++
星标
9
派生
0
PR 合并指标
30 天内没有已合并 PR

描述

currently background service holds a lot of memory , full memory which is inefficient
https://github.com/Agent-Hellboy/hcp/blob/6cc334d86d75df82f590c35a7c8c757347462f5e/src/main.cpp#L14

## Description

The clipboard history manager daemon should maintain an in-memory index (memtable) for fast reads and writes, following a Log-Structured Merge Tree (LSM Tree) architecture. This design allows the service to act like a lightweight embedded database, optimized for write-heavy workloads like clipboard updates.

---

## Goals

- Maintain a **memtable** (e.g., `std::map`, `unordered_map`, skip list) to hold the latest clipboard entries.
- Append every write to a **Write-Ahead Log (WAL)** for crash recovery.
- Periodically **flush memtable to SSTable** on disk when it reaches size limits.
- Support **tombstone deletion markers** for clipboard entries (useful for soft deletes and later compaction).
- Run background **compaction** to merge SSTables and remove stale data.

---

## Subtasks

- [ ] Implement in-memory `memtable` with sorted keys (e.g., timestamp or clipboard ID)
- [ ] Design and write to **WAL** on every insert/update/delete
- [ ] On daemon start, **rebuild memtable** from WAL
- [ ] Periodically flush memtable to disk as sorted immutable file (SSTable)
- [ ] Mark deleted entries with tombstones and skip them in reads
- [ ] Create background thread for compaction of old SSTables
- [ ] (Optional) Use Bloom filters or index summary for SSTable lookup optimization
- [ ] Add metrics/logs for memtable size, flushes, compaction status

---

## Notes

- This feature turns the daemon into a mini-LSM storage engine for clipboard history.
- All entries (text, image paths, etc.) can be indexed and persisted using this architecture.
- Crash recovery guarantees come from WAL.
- Inspired by RocksDB/LevelDB-style design, simplified for clipboard use case.

---

## References

- [LSM Tree – CMU 15-445 Notes](https://15721.courses.cs.cmu.edu/spring2019/slides/10-lsm.pdf)
- [WAL and SSTables – RocksDB Docs](https://rocksdb.org/blog/2019/01/08/wal.html)
- [Skip Lists vs B-trees](https://jepsen.io/analyses/skip-lists)

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。