matrix-org / matrix-org/rust-synapse-compress-state
Gracefully handle rooms where the compressor generates more rows than started with
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 173
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
I ran this against each room in my database to see what sort of changes it would want to make, and I'm wondering if it's normal for it to generate **more** rows than we started with?
For example:
```
Fetching state from DB for room '!redacted:matrix.org'...
Got initial state from database. Checking for any missing state groups...
Number of state groups: 108
Number of rows in current table: 187
Compressing state...
Number of rows after compression: 257 (137.43%)
Compression Statistics:
Number of forced resets due to lacking prev: 2
Number of compressed rows caused by the above: 23
Number of state groups changed: 12
Writing changes...
Checking that state maps match...
New state map matches old one
```
Full list, there are a couple others above 100%:
```
Number of rows after compression: 9 (100.00%)
Number of rows after compression: 8 (100.00%)
Number of rows after compression: 19 (54.29%)
Number of rows after compression: 26 (96.30%)
Number of rows after compression: 26 (86.67%)
Number of rows after compression: 27 (87.10%)
Number of rows after compression: 257 (137.43%)
Number of rows after compression: 7 (100.00%)
Number of rows after compression: 1292 (58.44%)
Number of rows after compression: 9 (100.00%)
Number of rows after compression: 37 (80.43%)
Number of rows after compression: 137 (125.69%)
Number of rows after compression: 851433 (20.22%)
Number of rows after compression: 27 (108.00%)
Number of rows after compression: 61136 (52.82%)
Number of rows after compression: 1708 (67.75%)
Number of rows after compression: 14 (100.00%)
Number of rows after compression: 6872 (26.59%)
Number of rows after compression: 76916 (99.81%)
Number of rows after compression: 18 (100.00%)
Number of rows after compression: 104922 (81.30%)
Number of rows after compression: 10786 (80.27%)
```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported compression output against a database containing the affected rooms, using the row counts and compression statistics in the issue as checks. Determine whether cases above 100% are expected or indicate a failure, then define completion as graceful, verified handling of those cases without an invalid state map.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100