element-hq / element-hq/synapse

Complement `TestRoomSummaryAllowedRoomIDs/restricted_room_includes_allowed_room_ids` is flaky

Closed
#19,905 3 comments 0 reactions 0 assignees View on GitHub
A-Room-Summary A-Testing A-Workers T-Defect Z-Read-After-Write
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

The `TestRoomSummaryAllowedRoomIDs/restricted_room_includes_allowed_room_ids` Complement test is flaky (test was added in https://github.com/matrix-org/complement/pull/873)

Added to https://github.com/element-hq/synapse/issues/18537 (list of known flaky tests)

Complement test source: [`tests/room_summary_test.go#L59-L69`](https://github.com/matrix-org/complement/blob/50044edb68c37c7ee4c1506f76d904d3d513fe2c/tests/room_summary_test.go#L59-L69)

Example: https://github.com/element-hq/synapse/actions/runs/28601133960/job/84838454090?pr=19871
```
❌ TestRoomSummaryAllowedRoomIDs/restricted_room_includes_allowed_room_ids (410ms)
room_summary_test.go:61: MatchResponse key 'join_rule' missing - http://127.0.0.1:32853/_matrix/client/v1/room_summary/%!l(MISSING)ZgQByxwZPjAuAnLnk:hs1 => {"room_id":"!lZgQByxwZPjAuAnLnk:hs1","room_version":"8","num_joined_members":1,"world_readable":false,"guest_can_join":false,"allowed_room_ids":["!NCUEzyGBEscdYvWgLu:hs1"],"membership":"join"}
```

### Investigation

Synapse currently pulls out the `join_rule` from the `room_stats_state` table but this is populated in the background as replication traffic reaches the worker that is configured to `run_background_tasks`.

https://github.com/element-hq/synapse/blob/5df6d1be652b46fc6201c9eb2541bf1828f83274/synapse/handlers/room_summary.py#L782-L804

https://github.com/element-hq/synapse/blob/5df6d1be652b46fc6201c9eb2541bf1828f83274/synapse/handlers/stats.py#L320

This means that a workerized deployment of Synapse will have some delay in the room summary being correct after the state changes. Currently, the Complement test expects things to be updated immediately.

What's extra strange is that `allowed_room_ids` appears in the `/room_summary` response but not the `join_rule`. This is because we look at the actual `m.room.join_rule` state event in the room for `allowed_room_ids` but the stats for the `join_rule`. Since, we have to fetch the `m.room.join_rule` state event in some cases, we could just always pull from that source of truth instead. This would fix this particular test 💪

Although, there is a bigger question around how the Complement test expects the `/room_summary` endpoint to give accurate results immediately. And is another example of a read-after-write inconsistency problem in Synapse.

Contributor guide

Open the contributing guide

Research direction

Start with synapse/handlers/room_summary.py and synapse/handlers/stats.py, then inspect Complement's tests/room_summary_test.go around lines 59-69 and the linked flaky run. Determine the intended read-after-write behavior for join_rule and room summaries in workerized deployments. Done means the behavior is consistent with that decision and the Complement test no longer flakes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, python
Domain
api, backend, distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.