element-hq / element-hq/synapse

Synapse decided I'm no longer a member of a room

Open
#15,258 0 comments 0 reactions 0 assignees View on GitHub
A-Membership A-State-Resolution O-Uncommon S-Minor T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#15258](https://github.com/matrix-org/synapse/issues/15258).

---

### Description

So the story starts at a room I'm in (PG Lounge) started missing a lot of peoples messages. I wrote it off as some synapse bug, and proceeded to just purge the room and rejoin. This seemed to fix it for about 4 days. This morning, Synapse decided that I am no longer a member of the room. Upon rejoining, most messages are missing and the timeline is messed up:

![image](https://user-images.githubusercontent.com/8676443/224649283-ad5f805e-b999-4718-9c53-54ff39b15ee2.png)

### Steps to reproduce

I don't know.

### Homeserver

namazso.eu

### Synapse Version

1.78.0

### Installation Method

Docker (matrixdotorg/synapse)

### Database

PostgreSQL 12. Single server. Yes, about 2 years ago. No.

### Workers

Single process

### Platform

Probably not relevant, but Fedora IoT, official docker container.

### Configuration

Presence is enabled.

### Relevant log output

```shell
2023-03-13 06:18:55,764 - synapse.federation.sender.per_destination_queue - 561 - INFO - federation_transaction_transmission_loop-314815 - Catching up rooms to conorz.tk: '!xGDfMAzPVtiqAxdQIu:privacyguides.org'
2023-03-13 06:21:52,697 - synapse.handlers.typing - 373 - INFO - PUT-148747 - Got typing update from @123bob123:matrix.org: {'room_id': '!xGDfMAzPVtiqAxdQIu:privacyguides.org', 'typing': True, 'user_id': '@123bob123:matrix.org'}
2023-03-13 06:22:02,693 - synapse.handlers.typing - 373 - INFO - PUT-148750 - Got typing update from @123bob123:matrix.org: {'room_id': '!xGDfMAzPVtiqAxdQIu:privacyguides.org', 'typing': False, 'user_id': '@123bob123:matrix.org'}
2023-03-13 06:22:06,687 - synapse.handlers.typing - 373 - INFO - PUT-148752 - Got typing update from @123bob123:matrix.org: {'room_id': '!xGDfMAzPVtiqAxdQIu:privacyguides.org', 'typing': True, 'user_id': '@123bob123:matrix.org'}
2023-03-13 06:22:13,948 - synapse.handlers.typing - 373 - INFO - PUT-148759 - Got typing update from @123bob123:matrix.org: {'room_id': '!xGDfMAzPVtiqAxdQIu:privacyguides.org', 'typing': False, 'user_id': '@123bob123:matrix.org'}
2023-03-13 06:22:14,702 - synapse.federation.federation_server - 1236 - INFO - _process_incoming_pdus_in_room_inner-6145 - handling received PDU in room !xGDfMAzPVtiqAxdQIu:privacyguides.org:
2023-03-13 06:22:17,692 - synapse.handlers.typing - 373 - INFO - PUT-148761 - Got typing update from @123bob123:matrix.org: {'room_id': '!xGDfMAzPVtiqAxdQIu:privacyguides.org', 'typing': True, 'user_id': '@123bob123:matrix.org'}
2023-03-13 06:22:19,910 - synapse.handlers.typing - 373 - INFO - PUT-148762 - Got typing update from @123bob123:matrix.org: {'room_id': '!xGDfMAzPVtiqAxdQIu:privacyguides.org', 'typing': False, 'user_id': '@123bob123:matrix.org'}
2023-03-13 06:22:20,145 - synapse.federation.federation_server - 1236 - INFO - _process_incoming_pdus_in_room_inner-6146 - handling received PDU in room !xGDfMAzPVtiqAxdQIu:privacyguides.org:
2023-03-13 06:24:54,323 - synapse.federation.sender.per_destination_queue - 561 - INFO - federation_transaction_transmission_loop-317840 - Catching up rooms to projectsegfau.lt: '!xGDfMAzPVtiqAxdQIu:privacyguides.org'
2023-03-13 06:39:21,005 - synapse.federation.federation_server - 1236 - INFO - _process_incoming_pdus_in_room_inner-6168 - handling received PDU in room !xGDfMAzPVtiqAxdQIu:privacyguides.org:
2023-03-13 06:49:57,469 - synapse.federation.sender.per_destination_queue - 561 - INFO - federation_transaction_transmission_loop-317856 - Catching up rooms to projectsegfau.lt: '!xGDfMAzPVtiqAxdQIu:privacyguides.org'
2023-03-13 07:14:57,581 - synapse.federation.sender.per_destination_queue - 561 - INFO - federation_transaction_transmission_loop-322628 - Catching up rooms to projectsegfau.lt: '!xGDfMAzPVtiqAxdQIu:privacyguides.org'
2023-03-13 07:32:02,394 - synapse.federation.sender.per_destination_queue - 561 - INFO - federation_transaction_transmission_loop-322712 - Catching up rooms to conorz.tk: '!xGDfMAzPVtiqAxdQIu:privacyguides.org'
2023-03-13 07:39:57,554 - synapse.federation.sender.per_destination_queue - 561 - INFO - federation_transaction_transmission_loop-325709 - Catching up rooms to projectsegfau.lt: '!xGDfMAzPVtiqAxdQIu:privacyguides.org'
2023-03-13 08:05:00,869 - synapse.federation.sender.per_destination_queue - 561 - INFO - federation_transaction_transmission_loop-327248 - Catching up rooms to projectsegfau.lt: '!xGDfMAzPVtiqAxdQIu:privacyguides.org'
2023-03-13 08:07:55,540 - synapse.federation.federation_server - 1236 - INFO - _process_incoming_pdus_in_room_inner-6243 - handling received PDU in room !xGDfMAzPVtiqAxdQIu:privacyguides.org:
2023-03-13 08:08:35,089 - synapse.state - 656 - INFO - persist_events-11923 - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259202, 1259266]
2023-03-13 08:08:36,102 - synapse.state - 656 - INFO - persist_events-11925 - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259202, 1259267, 1259266]
2023-03-13 08:08:38,060 - synapse.state - 656 - INFO - persist_events-11927 - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259202, 1259267, 1259268, 1259266]
2023-03-13 08:08:45,513 - synapse.state - 656 - INFO - persist_events-11940 - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259202, 1259267, 1259266, 1259268, 1259269]
2023-03-13 08:08:46,793 - synapse.state - 656 - INFO - persist_events-11942 - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259202, 1259267, 1259266, 1259268, 1259269, 1259271]
2023-03-13 08:08:47,060 - synapse.handlers.sync - 1373 - INFO - GET-152610 - User membership change between getting rooms and current token: @namazso:namazso.eu leave !xGDfMAzPVtiqAxdQIu:privacyguides.org
2023-03-13 08:09:52,837 - synapse.federation.federation_server - 1236 - INFO - _process_incoming_pdus_in_room_inner-6251 - handling received PDU in room !xGDfMAzPVtiqAxdQIu:privacyguides.org:
2023-03-13 08:10:12,257 - synapse.state - 656 - INFO - GET-152740-$tAUp2CEzV6UVgnnyP0KtsnLnM48B-SGXWt6-InGOWOk - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259275, 1259276]
2023-03-13 08:10:20,674 - synapse.handlers.sync - 1373 - INFO - GET-152839 - User membership change between getting rooms and current token: @namazso:namazso.eu join !xGDfMAzPVtiqAxdQIu:privacyguides.org
2023-03-13 08:10:30,834 - synapse.state - 656 - INFO - persist_events-12056 - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259283, 1259278]
2023-03-13 08:10:38,105 - synapse.state - 656 - INFO - persist_events-12067 - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259283, 1259284, 1259278]
2023-03-13 08:10:45,766 - synapse.state - 656 - INFO - persist_events-12079 - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259283, 1259284, 1259285, 1259278]
2023-03-13 08:11:15,155 - synapse.state - 656 - INFO - persist_events-12086 - Resolving state for !xGDfMAzPVtiqAxdQIu:privacyguides.org with groups [1259278, 1259283, 1259284, 1259285, 1259286]
```

### Anything else that would be useful to know?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing the logged entry points in synapse.federation.federation_server, synapse.state, and synapse.handlers.sync, focusing on state resolution and the reported leave/join membership changes. There is no reproduction or test named in the issue; done would mean identifying the cause of the incorrect membership and adding a regression test that preserves the room timeline.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, postgresql, python
Domain
backend, distributed-systems, networking
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.