bitemyapp / bitemyapp/mentats

Streaming/incremental checkpointing in NockApps

Open
#14 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
7
Forks
0
PR merge metrics
No merged PRs in 30d

Description

This could straightforwardly build on #13. Currently we're copying the slab, so that doubles the peak memory usage, then we're batch-jamming the slab into a jam copy of the slab, that puts us at 3x memory usage temporarily until the save finishes and the slab + jam deallocate. This should be avoidable. It's possible that after Habit/Chaff are done and have both jam and cue implemented that we could make a streaming or incremental byte-oriented interface for progressively deallocating the slab as we fill in the jammed representation. Not super critical but a nice to have. The need for this probably goes away for now if we implement PMA/WAL/CDC for NockVM but this is easier and quicker to do.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with issue #13 and trace the current slab-copying checkpoint path; this issue names no files, tests, or entry points. Establish whether streaming or incremental byte-oriented checkpointing is still needed after Habit/Chaff and possible PMA/WAL/CDC work, then define completion as avoiding the temporary 3x memory usage during save.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.