matrix-org / matrix-org/matrix-rust-sdk
event cache: be more conservative when adding duplicated events to a linked chunk
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 2.3k
- Forks
- 500
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 106
Description
When we're adding new events into the linked chunk, we try to minimize storing of back-pagination prev-batch tokens:
- when pushing to the tail, if all events have been duplicated, independently of ordering, we won't deduplicate/remove them, and we'll not store the prev-batch token.
- when inserting a chunk of events from a recent back-pagination, and they've all been deduplicated, we won't deduplicate/remove them, and we'll not store the prev-batch token.
Unfortunately, this is not as simple as that: we should also check the orderings of the events that have been deduplicated, in particular that they're indeed ordered the same way they're ordered in the linked chunk representation. If that's not the case, then we should keep the prev-batch token and keep on iterating until we get the same events in the same ordering.
Related to #3280.
cc @Hywan
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the event-cache logic that adds events to linked chunks, including tail pushes and recent back-pagination insertion. Compare cases where duplicated events have different orderings, and verify completion when the implementation preserves the prev-batch token until matching event order is reached.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100