Collaboration Y.Doc document-state table grows unbounded
Nobody has claimed this yet.
- Dominant language
- Elixir
- Stars
- 296
- Forks
- 86
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 50
Description
What's broken
collaboration_document_states has grown to about 1 GB and keeps growing, almost entirely stale update rows that never get compacted or pruned.
Pruning only runs in-process while a document has an active session, and only deletes update rows once a checkpoint exists for that document. Checkpoints are only created after 500 updates, which most documents never reach, so they never get a checkpoint and never get pruned. Nothing sweeps documents with no active session, so once a room closes its rows sit there indefinitely.
Evidence
- About 1,062 MB.
- 5,893 documents, 123,727 update rows, 605 checkpoints.
- 97% of documents (5,738) have no checkpoint.
- 99% of update rows (122,968) are older than a day and still present.
Why it matters
It is roughly five times the size of workflow_snapshots and grows with editor usage. Update rows are only needed to reconstruct an open document; once a room closes they can be compacted to a single state or dropped.
What to fix
A background job that compacts and prunes across all documents, not just the ones currently open. Some combination of: compact to a checkpoint on unbind when the last user leaves, a scheduled sweep that checkpoints and prunes documents carrying many stale updates, and a lower checkpoint threshold so compaction actually fires during normal use. Retain enough recent state to reconstruct any document that gets reopened.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the collaboration_document_states lifecycle, checkpoint creation threshold, and in-process pruning described in the issue. Examine what happens when the last user leaves a room and how closed documents could be revisited. Done means stale update rows are compacted or pruned across inactive documents while reopened documents remain reconstructable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100