matrix-org / matrix-org/complement

Test failures when v12 rooms are the default

Open
#799 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
99
Forks
72
Avg merge
4d 1h
Merged PRs (30d)
8

Description

## Failing tests on v12

### `tests/csapi`

Most of these tests directly do something with PLs causing them to break. Some weird quirks with sync though need investigation.

```
--- FAIL: TestRoomDeleteAlias (8.40s)
--- FAIL: TestRoomDeleteAlias/Parallel (0.00s)
--- FAIL: TestRoomDeleteAlias/Parallel/Users_with_sufficient_power-level_can_delete_other's_aliases (0.70s)
--- FAIL: TestRoomDeleteAlias/Parallel/Regular_users_can_add_and_delete_aliases_when_m.room.aliases_is_restricted (0.88s)

--- FAIL: TestRoomMembers (8.35s)
--- FAIL: TestRoomMembers/Parallel (0.00s)
--- FAIL: TestRoomMembers/Parallel/Test_that_we_can_be_reinvited_to_a_room_we_created (0.52s)

--- FAIL: TestDemotingUsersViaUsersDefault (7.36s)

--- FAIL: TestPowerLevels (7.51s)
--- FAIL: TestPowerLevels/GET_/rooms/:room_id/state/m.room.power_levels_can_fetch_levels (0.01s)
--- FAIL: TestPowerLevels/PUT_/rooms/:room_id/state/m.room.power_levels_can_set_levels (0.02s)
--- FAIL: TestPowerLevels/PUT_power_levels_should_not_explode_if_the_old_power_levels_were_empty (0.01s)

--- FAIL: TestRoomCreationReportsEventsToMyself (12.77s)
--- FAIL: TestRoomCreationReportsEventsToMyself/parallel (0.00s)
--- FAIL: TestRoomCreationReportsEventsToMyself/parallel/Room_creation_reports_m.room.create_to_myself (0.12s)

--- FAIL: TestSync (20.19s)
--- FAIL: TestSync/parallel (2.87s)
--- FAIL: TestSync/parallel/sync_should_succeed_even_if_the_sync_token_points_to_a_redaction_of_an_unknown_event (10.18s)

--- FAIL: TestSyncTimelineGap (18.62s)
--- FAIL: TestSyncTimelineGap/incremental (5.12s)
--- FAIL: TestSyncTimelineGap/full (5.16s)
```

### `tests`

Basically any test that uses the complement federation server is likely to be affected by this.

```
--- FAIL: TestIsDirectFlagFederation (7.66s)
--- FAIL: TestEventAuth (7.88s)
--- FAIL: TestFederationRedactSendsWithoutEvent (10.49s)
--- FAIL: TestInboundFederationRejectsEventsWithRejectedAuthEvents (13.96s)
--- FAIL: TestGetMissingEventsGapFilling (13.77s)
--- FAIL: TestOutboundFederationIgnoresMissingEventWithBadJSONForRoomVersion6 (8.16s)
--- FAIL: TestOutboundFederationEventSizeGetMissingEvents (8.14s)
--- FAIL: TestJoinViaRoomIDAndServerName (8.97s)
--- FAIL: TestJoinFederatedRoomWithUnverifiableEvents (7.96s)
--- FAIL: TestJoinFederatedRoomWithUnverifiableEvents//send_join_response_missing_signatures_shouldn't_block_room_join (0.00s)
--- FAIL: TestJoinFederatedRoomWithUnverifiableEvents//send_join_response_with_bad_signatures_shouldn't_block_room_join (0.00s)
--- FAIL: TestJoinFederatedRoomWithUnverifiableEvents//send_join_response_with_unobtainable_keys_shouldn't_block_room_join (0.00s)
--- FAIL: TestJoinFederatedRoomWithUnverifiableEvents//send_join_response_with_state_with_unverifiable_auth_events_shouldn't_block_room_join (0.00s)
--- FAIL: TestOutboundFederationSend (7.61s)
--- FAIL: TestNetworkPartitionOrdering (7.85s)
--- FAIL: TestSyncOmitsStateChangeOnFilteredEvents (8.16s)
--- FAIL: TestUnrejectRejectedEvents (7.92s)
```

Contributor guide

Open the contributing guide

Research direction

Start by running the failing tests listed under tests/csapi and tests, focusing on cases involving power levels, sync, room creation, and federation. Compare failures with v12 rooms as the default and trace the Complement federation server interactions. Done means the listed failures pass with v12 as the default room version.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.