matrix-org/sytest

View on GitHub

Black-box integration testing for Matrix homeservers

Stars: 81
Forks: 63
Open beginner issues: 0
Indexed issues: 80
Avg merge: 1h 47m
Merged PRs (30d): 1

Dominant language: Perl
License: Apache-2.0
Last GitHub push: Sep 15, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: good first issue help wanted

Beginner friendly Everything

Open All

80 open issues indexed

Sytest fails if only room version 9 and 10 is allowed Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

matrix-org/sytest#1393 · Opened Jan 9, 2025
`After /purge_history users still get pushed for new messages` flake Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 35/100

matrix-org/sytest#1368 · 6 comments · Opened Sep 4, 2023
Flake: If a device list update goes missing, the server resyncs on the next one Open

Difficulty 3/5 1-2 days Newbie friendliness 20/100

matrix-org/sytest#1364 · 7 comments · Opened Jul 31, 2023
Occsionally the nightly docker build job times out Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 35/100

matrix-org/sytest#1351 · Opened May 18, 2023
Docker build failing again due to codecov Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

matrix-org/sytest#1348 · Opened Apr 13, 2023
`Can receive redactions from regular users over federation in room version 3` flake? Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 35/100

matrix-org/sytest#1329 · Opened Jan 17, 2023
`Can receive redactions from regular users over federation in room version` is flakey Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 20/100

matrix-org/sytest#1322 · 1 comment · Opened Dec 5, 2022
`/upgrade preserves direct room state` is flakey Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 48/100

matrix-org/sytest#1321 · Opened Dec 2, 2022
`Changing user-signing key notifies local users` is flaky Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 42/100

matrix-org/sytest#1318 · Opened Nov 15, 2022
`Local delete device changes appear in v2 /sync` is flaky Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1317 · Opened Nov 15, 2022
30rooms/60version_upgrade: "$user_type user has push rules copied to upgraded room" does not properly fail on 0 return Open

T-Defect

Difficulty 3/5 1-2 days Newbie friendliness 38/100

matrix-org/sytest#1314 · 2 comments · Opened Nov 6, 2022
31sync/09archived: "Newly left rooms appear in the leave section of gapped sync": possible bug in not creating the filter properly Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

matrix-org/sytest#1309 · Opened Oct 21, 2022
31sync/10archived-ban: "Newly banned rooms appear in the leave section of incremental sync" is defined twice Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

matrix-org/sytest#1305 · Opened Oct 14, 2022
Sytest is slow and takes 4 minutes to run a single test Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

matrix-org/sytest#1299 · 3 comments · Opened Oct 5, 2022
Suspected flake: `Backfill works correctly with history visibility set to joined` Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1295 · Opened Sep 27, 2022
Suspected flake: `All room members see all room members' presence in global initialSync` Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 25/100

matrix-org/sytest#1294 · Opened Sep 27, 2022
Device list tests do not take "encrypted" property of a room into account Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

matrix-org/sytest#1280 · Opened Aug 18, 2022
Suspected Flake: `User sees updates to presence from other users in the incremental sync` Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 25/100

matrix-org/sytest#1279 · 2 comments · Opened Aug 10, 2022
Suspected flake: `Can recv device messages over federation` Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 20/100

matrix-org/sytest#1277 · 1 comment · Opened Jul 21, 2022
Suspected flake: `Presence changes to UNAVAILABLE are reported to remote room members` Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 20/100

matrix-org/sytest#1276 · 1 comment · Opened Jul 21, 2022
Suspected flake: `Typing notifications also sent to remote room members` Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 25/100

matrix-org/sytest#1275 · 1 comment · Opened Jul 21, 2022
Suspected flake: `Room state after a rejected message event is the same as before` Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 30/100

matrix-org/sytest#1273 · Opened Jul 19, 2022
Suspected flake: ` Rejecting invite over federation doesn't break incremental /sync` Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 20/100

matrix-org/sytest#1268 · Opened Jul 12, 2022
Suspected flake: `Can fetch images in room` Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 25/100

matrix-org/sytest#1258 · Opened Jun 9, 2022
Suspected flake: `Remote banned user is kicked and may not rejoin until unbanned` Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 20/100

matrix-org/sytest#1257 · 1 comment · Opened Jun 7, 2022
`31room-messages.pl` lazy-loading test is probably incorrect/too specific to one bug Open

T-Enhancement

Difficulty 3/5 1-2 days Newbie friendliness 48/100

matrix-org/sytest#1208 · Opened Mar 6, 2022
`Can invite unbound 3pid over federation with users from both servers` is flaky Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 35/100

matrix-org/sytest#1187 · 1 comment · Opened Feb 11, 2022
`/whois` is flakey(?) Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1180 · 2 comments · Opened Dec 10, 2021
When bootstrapping, check the user's namespace first with their own branch before falling back on matrix-org Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

matrix-org/sytest#1173 · 2 comments · 1 reaction · Opened Nov 29, 2021
Suspected flakiness in `New federated private chats get full presence information (SYN-115)` Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 30/100

matrix-org/sytest#1164 · 9 comments · Opened Nov 1, 2021
"Existing members see new member's presence" is flaky Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1157 · 3 comments · Opened Oct 21, 2021
`Device deletion propagates over federation` is flaky Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 25/100

matrix-org/sytest#1156 · 2 comments · Opened Oct 13, 2021
sytest won't warn you if you ask it to run tests in a file that doesn't exist Open

Difficulty 2/5 1-3 hours Newbie friendliness 52/100

matrix-org/sytest#1152 · 1 comment · Opened Oct 7, 2021
`/joined_rooms returns only joined rooms` is flakey Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 25/100

matrix-org/sytest#1151 · 2 comments · Opened Oct 7, 2021
`Server correctly handles incoming m.device_list_update` is flaky Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 20/100

matrix-org/sytest#1137 · 3 comments · Opened Sep 13, 2021
`Device list doesn't change if remote server is down` is flaky Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1136 · 16 comments · Opened Sep 10, 2021
`If remote user leaves room, changes device and rejoins we see update in /keys/changes` is flaky Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 28/100

matrix-org/sytest#1130 · 9 comments · Opened Sep 6, 2021
`Get presence for newly joined members in incremental sync` is flakey Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1125 · 2 comments · Opened Aug 23, 2021
`If remote user leaves room, changes device and rejoins we see update in sync` is flaky Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1120 · 6 comments · Opened Aug 17, 2021
"GET /capabilities is present and well formed for registered user" requires non-required fields Open

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

matrix-org/sytest#1074 · Opened Jul 11, 2021
"POST /join/:room_id can join a room with custom content" sends unexpected parameters. Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1073 · Opened Jul 11, 2021
Test "GET /rooms/:room_id/joined_members fetches my membership" expects display_name and avatar_url to be present Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

matrix-org/sytest#1072 · Opened Jul 11, 2021
The "m.room.power_levels fetches powerlevels" test checks for non-required fields Open

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

matrix-org/sytest#1070 · 1 comment · Opened Jul 10, 2021
Unknown "GET /rooms/:room_id/state/m.room.member/:user_id?format=event" API Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

matrix-org/sytest#1069 · Opened Jul 10, 2021
`GET /publicRooms lists rooms` is flakey Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 20/100

matrix-org/sytest#1051 · Opened May 27, 2021
`Newly joined room includes presence in incremental sync` is flakey Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 20/100

matrix-org/sytest#1049 · 3 comments · Opened May 24, 2021
`Remote user can backfill in a room with version N` is flaky Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 25/100

matrix-org/sytest#1042 · 1 comment · Opened May 13, 2021
`If remote user leaves room we no longer receive device updates` is flakey Open

Z-Flaky

Difficulty 4/5 3-5 days Newbie friendliness 25/100

matrix-org/sytest#1036 · 5 comments · Opened Apr 22, 2021
`Multiple calls to /sync should not cause 500 errors` is flakey Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1035 · Opened Apr 22, 2021
`Can backfill purged history` is flakey Open

Z-Flaky

Difficulty 3/5 1-2 days Newbie friendliness 25/100

matrix-org/sytest#1033 · 3 comments · Opened Apr 21, 2021

matrix-org/sytest

Get new issues in your inbox