dimensionalOS / dimensionalOS/dimos
Batch high frequency memory2 messages
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.5k
- Forks
- 808
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 233
Description
The ZED Mini's IMU runs at 800Hz (we don't ingest it yet, but that's the rate we need to support), and at those rates the rapid locking/unlocking of the database per memory stream means a high-hertz stream builds up back pressure and blocks other streams from writing. Batching the writes into one transaction instead of a lock+commit per message fixes it.
Synced from DIM-1457 by jeff
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the database write path for memory streams and inspect where each message acquires a lock and commits. Reproduce or trace behavior with a high-frequency stream, then verify that writes are batched into one transaction and no longer create back pressure that blocks other streams.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- database, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100