matrix-org / matrix-org/rust-synapse-compress-state
Incremental / partial compression
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 173
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
Right now, compressing a room requires holding its entire state in RAM, which can become arbitrarily large.
I don't know the internals of the compression algorithm, but given that state groups seem to be replaced one by one in an idempotent fashion, would it be possible to compress a room only partially somehow? Perhaps only the first N percent of rows, and do it a couple of times? Or maybe have multiple iterations of load, compress, free?
That would be tremendously helpful 👍
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the compression algorithm and its handling of state groups, focusing on why the full room state must remain in RAM. Evaluate whether partial row processing or repeated load, compress, and free iterations can preserve the existing idempotent behavior. Done means a room can be compressed with a bounded memory requirement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100