element-hq / element-hq/synapse

Complement `TestThreadReceiptsInSyncMSC4102` is flaky

Open
#19,171 2 comments 0 reactions 0 assignees View on GitHub
A-Read-Receipts A-Testing A-Threads T-Other
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

As already tracked in our [list of flaky tests](https://github.com/element-hq/synapse/issues/18537), the [`TestThreadReceiptsInSyncMSC4102`](https://github.com/matrix-org/complement/blob/1de64122c351539624c943d0ad7433e7b75e1ae0/tests/csapi/thread_notifications_test.go#L323-L376) Complement test is flaky when Synapse is run with workers.

We're also running into this flaky failure in the CI for our [Synapse Rust apps](https://github.com/element-hq/synapse-rust-apps) which is what [caused](https://github.com/element-hq/synapse-rust-apps/pull/336#discussion_r2515632199) me also look into this.

### Test logs

From the test logs, we can see that the remote federated server sees the read receipt for the thread but not the unthreaded receipt as expected. Per the comment in the test, the remote federated server should see 1) only the unthreaded receipt or b) both threaded and unthreaded if they end up being sent as separate EDUs.

The [Complement test](https://github.com/matrix-org/complement/blob/1de64122c351539624c943d0ad7433e7b75e1ae0/tests/csapi/thread_notifications_test.go#L323-L376) itself looks pretty sound and this is probably a Synapse bug.

```
##[group]❌ TestThreadReceiptsInSyncMSC4102 (12.14s)
client.go:805: [CSAPI] POST hs1/_matrix/client/v3/register => 200 OK (58.500018ms)
client.go:805: [CSAPI] GET hs1/_matrix/client/v3/pushrules/global/postcontent/.io.element.msc4306.rule.subscribed_thread => 200 OK (7.414187ms)
client.go:805: [CSAPI] PUT hs1/_matrix/client/v3/pushrules/global/postcontent/.io.element.msc4306.rule.subscribed_thread/enabled => 200 OK (12.681882ms)
client.go:805: [CSAPI] GET hs1/_matrix/client/v3/pushrules/global/postcontent/.io.element.msc4306.rule.unsubscribed_thread => 200 OK (4.413429ms)
client.go:805: [CSAPI] PUT hs1/_matrix/client/v3/pushrules/global/postcontent/.io.element.msc4306.rule.unsubscribed_thread/enabled => 200 OK (7.22621ms)
client.go:805: [CSAPI] POST hs2/_matrix/client/v3/register => 200 OK (61.749714ms)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/pushrules/global/postcontent/.io.element.msc4306.rule.subscribed_thread => 200 OK (7.271153ms)
client.go:805: [CSAPI] PUT hs2/_matrix/client/v3/pushrules/global/postcontent/.io.element.msc4306.rule.subscribed_thread/enabled => 200 OK (6.592844ms)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/pushrules/global/postcontent/.io.element.msc4306.rule.unsubscribed_thread => 200 OK (15.268867ms)
client.go:805: [CSAPI] PUT hs2/_matrix/client/v3/pushrules/global/postcontent/.io.element.msc4306.rule.unsubscribed_thread/enabled => 200 OK (10.124181ms)
client.go:805: [CSAPI] POST hs1/_matrix/client/v3/createRoom => 200 OK (240.296146ms)
client.go:805: [CSAPI] POST hs2/_matrix/client/v3/join/!rkvKOcpUehaYxuerxr:hs1 => 200 OK (414.652287ms)
client.go:805: [CSAPI] PUT hs1/_matrix/client/v3/rooms/!rkvKOcpUehaYxuerxr:hs1/send/m.room.message/1 => 200 OK (80.06651ms)
thread_notifications_test.go:339: SendEventSynced waiting for event ID $chL6ViiGN82z1AG_HYNpBHu5Kz1d7gitfQcgjatNM4o
client.go:805: [CSAPI] GET hs1/_matrix/client/v3/sync => 200 OK (87.664041ms)
client.go:805: [CSAPI] PUT hs1/_matrix/client/v3/rooms/!rkvKOcpUehaYxuerxr:hs1/send/m.room.message/2 => 200 OK (55.270868ms)
thread_notifications_test.go:346: SendEventSynced waiting for event ID $ybQ1MPrWa-s5Equ-NktU3LMrqUIpE1Nej46GzbkrFIY
client.go:805: [CSAPI] GET hs1/_matrix/client/v3/sync => 200 OK (22.08896ms)
client.go:805: [CSAPI] POST hs1/_matrix/client/v3/rooms/!rkvKOcpUehaYxuerxr:hs1/receipt/m.read/$ybQ1MPrWa-s5Equ-NktU3LMrqUIpE1Nej46GzbkrFIY => 200 OK (26.900954ms)
client.go:805: [CSAPI] POST hs1/_matrix/client/v3/rooms/!rkvKOcpUehaYxuerxr:hs1/receipt/m.read/$ybQ1MPrWa-s5Equ-NktU3LMrqUIpE1Nej46GzbkrFIY => 200 OK (22.266235ms)
client.go:805: [CSAPI] GET hs1/_matrix/client/v3/sync => 200 OK (34.924612ms)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (150.036075ms)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (66.909635ms)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (384.574298ms)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (103.927914ms)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (261.148361ms)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (1.007134004s)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (1.007341932s)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (1.006467462s)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (1.006372485s)
client.go:805: [CSAPI] GET hs2/_matrix/client/v3/sync => 200 OK (1.007079951s)
thread_notifications_test.go:370: @user-167:hs2 MustSyncUntil: timed out after 6.004202564s. Seen 10 /sync responses. Checkers:
[t=151.041456ms] Response #1: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): Key rooms.join.!rkvKOcpUehaYxuerxr:hs1.ephemeral.events does not exist
[t=218.125397ms] Response #2: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): Key rooms.join.!rkvKOcpUehaYxuerxr:hs1.ephemeral.events does not exist
[t=603.097719ms] Response #3: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): check function did not pass while iterating over 1 elements: [{"type":"m.receipt","content":{"$ybQ1MPrWa-s5Equ-NktU3LMrqUIpE1Nej46GzbkrFIY":{"m.read":{"@user-166:hs1":{"thread_id":"$chL6ViiGN82z1AG_HYNpBHu5Kz1d7gitfQcgjatNM4o","ts":1762815095353}}}}}]
[t=707.17385ms] Response #4: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): check function did not pass while iterating over 0 elements: []
[t=968.629677ms] Response #5: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): check function did not pass while iterating over 0 elements: []
[t=1.976254907s] Response #6: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): Key rooms.join.!rkvKOcpUehaYxuerxr:hs1.ephemeral.events does not exist
[t=2.983737412s] Response #7: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): Key rooms.join.!rkvKOcpUehaYxuerxr:hs1.ephemeral.events does not exist
[t=3.990353693s] Response #8: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): Key rooms.join.!rkvKOcpUehaYxuerxr:hs1.ephemeral.events does not exist
[t=4.99690731s] Response #9: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): Key rooms.join.!rkvKOcpUehaYxuerxr:hs1.ephemeral.events does not exist
[t=6.00420064s] Response #10: SyncEphemeralHas(!rkvKOcpUehaYxuerxr:hs1): Key rooms.join.!rkvKOcpUehaYxuerxr:hs1.ephemeral.events does not exist,

2025/11/10 22:51:41 ============================================
```

### Dev notes

`TestThreadReceiptsInSyncMSC4102` was introduced in https://github.com/matrix-org/complement/pull/709

The Synapse "fix" was introduced in https://github.com/element-hq/synapse/pull/16927

Related [spec issue #1727](https://github.com/matrix-org/matrix-spec/issues/1727) and [MSC4102](https://github.com/matrix-org/matrix-spec-proposals/pull/4102) for clarifying the previous undefined behavior.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.