Agent-Hellboy / Agent-Hellboy/hcp
In-Memory LSM Tree (Memtable) Support in Clipboard Daemon
- 主要语言
- C++
- 星标
- 9
- 派生
- 0
- PR 合并指标
- 30 天内没有已合并 PR
描述
currently background service holds a lot of memory , full memory which is inefficient
https://github.com/Agent-Hellboy/hcp/blob/6cc334d86d75df82f590c35a7c8c757347462f5e/src/main.cpp#L14
## Description
The clipboard history manager daemon should maintain an in-memory index (memtable) for fast reads and writes, following a Log-Structured Merge Tree (LSM Tree) architecture. This design allows the service to act like a lightweight embedded database, optimized for write-heavy workloads like clipboard updates.
---
## Goals
- Maintain a **memtable** (e.g., `std::map`, `unordered_map`, skip list) to hold the latest clipboard entries.
- Append every write to a **Write-Ahead Log (WAL)** for crash recovery.
- Periodically **flush memtable to SSTable** on disk when it reaches size limits.
- Support **tombstone deletion markers** for clipboard entries (useful for soft deletes and later compaction).
- Run background **compaction** to merge SSTables and remove stale data.
---
## Subtasks
- [ ] Implement in-memory `memtable` with sorted keys (e.g., timestamp or clipboard ID)
- [ ] Design and write to **WAL** on every insert/update/delete
- [ ] On daemon start, **rebuild memtable** from WAL
- [ ] Periodically flush memtable to disk as sorted immutable file (SSTable)
- [ ] Mark deleted entries with tombstones and skip them in reads
- [ ] Create background thread for compaction of old SSTables
- [ ] (Optional) Use Bloom filters or index summary for SSTable lookup optimization
- [ ] Add metrics/logs for memtable size, flushes, compaction status
---
## Notes
- This feature turns the daemon into a mini-LSM storage engine for clipboard history.
- All entries (text, image paths, etc.) can be indexed and persisted using this architecture.
- Crash recovery guarantees come from WAL.
- Inspired by RocksDB/LevelDB-style design, simplified for clipboard use case.
---
## References
- [LSM Tree – CMU 15-445 Notes](https://15721.courses.cs.cmu.edu/spring2019/slides/10-lsm.pdf)
- [WAL and SSTables – RocksDB Docs](https://rocksdb.org/blog/2019/01/08/wal.html)
- [Skip Lists vs B-trees](https://jepsen.io/analyses/skip-lists)
贡献指南
这个仓库没有索引到贡献指南
评估
这个 Issue 还没有评估数据。