element-hq / element-hq/synapse
Complement `TestRoomSummaryAllowedRoomIDs/restricted_room_includes_allowed_room_ids` is flaky
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
The `TestRoomSummaryAllowedRoomIDs/restricted_room_includes_allowed_room_ids` Complement test is flaky (test was added in https://github.com/matrix-org/complement/pull/873)
Added to https://github.com/element-hq/synapse/issues/18537 (list of known flaky tests)
Complement test source: [`tests/room_summary_test.go#L59-L69`](https://github.com/matrix-org/complement/blob/50044edb68c37c7ee4c1506f76d904d3d513fe2c/tests/room_summary_test.go#L59-L69)
Example: https://github.com/element-hq/synapse/actions/runs/28601133960/job/84838454090?pr=19871
```
❌ TestRoomSummaryAllowedRoomIDs/restricted_room_includes_allowed_room_ids (410ms)
room_summary_test.go:61: MatchResponse key 'join_rule' missing - http://127.0.0.1:32853/_matrix/client/v1/room_summary/%!l(MISSING)ZgQByxwZPjAuAnLnk:hs1 => {"room_id":"!lZgQByxwZPjAuAnLnk:hs1","room_version":"8","num_joined_members":1,"world_readable":false,"guest_can_join":false,"allowed_room_ids":["!NCUEzyGBEscdYvWgLu:hs1"],"membership":"join"}
```
### Investigation
Synapse currently pulls out the `join_rule` from the `room_stats_state` table but this is populated in the background as replication traffic reaches the worker that is configured to `run_background_tasks`.
https://github.com/element-hq/synapse/blob/5df6d1be652b46fc6201c9eb2541bf1828f83274/synapse/handlers/room_summary.py#L782-L804
https://github.com/element-hq/synapse/blob/5df6d1be652b46fc6201c9eb2541bf1828f83274/synapse/handlers/stats.py#L320
This means that a workerized deployment of Synapse will have some delay in the room summary being correct after the state changes. Currently, the Complement test expects things to be updated immediately.
What's extra strange is that `allowed_room_ids` appears in the `/room_summary` response but not the `join_rule`. This is because we look at the actual `m.room.join_rule` state event in the room for `allowed_room_ids` but the stats for the `join_rule`. Since, we have to fetch the `m.room.join_rule` state event in some cases, we could just always pull from that source of truth instead. This would fix this particular test 💪
Although, there is a bigger question around how the Complement test expects the `/room_summary` endpoint to give accurate results immediately. And is another example of a read-after-write inconsistency problem in Synapse.
Contributor guide
Research direction
Start with synapse/handlers/room_summary.py and synapse/handlers/stats.py, then inspect Complement's tests/room_summary_test.go around lines 59-69 and the linked flaky run. Determine the intended read-after-write behavior for join_rule and room summaries in workerized deployments. Done means the behavior is consistent with that decision and the Complement test no longer flakes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, python
- Domain
- api, backend, distributed-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100