element-hq / element-hq/synapse

Rooms created without any membership (only `m.room.create` event)

Open
#15,005 0 comments 0 reactions 0 assignees View on GitHub
A-Create-Room A-DAG O-Uncommon S-Tolerable T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#15005](https://github.com/matrix-org/synapse/issues/15005).

---

### Description

Trying to tie up any loose-ends with the Gitter import as part of the [Gitter migration process](https://github.com/vector-im/roadmap/issues/26) and noticing that we have some Matrix rooms recorded that seem to broken on the Synapse side. Some of these rooms only have a `m.room.create` event in it while others have no events which means we can't interact with it at all since permission denied errors everywhere (no membership or power-level events).

~~The only way we recorded this room on the Gitter side is if we got a 200-response from Synapse originally when it was created. If Synapse is returning with a error-free response, there should be much better guarantees on the room being created successfully (atomic transaction). Of course it's possible this could have been corrupted after the fact though.~~ What most likely happened was the rooms were attempted to be created but timed out. Then we introduced code to reconcile the case where a Matrix room alias exists but we don't have the bridged Matrix room entry recorded, see https://github.com/matrix-org/synapse/issues/15005#issuecomment-1421481395

They all seem to be created on January 10th, 2023 which is probably in the middle of importing messages and the import scripts.

Here is the list of affected rooms that I know of:

- `!gHbYoGxiEuXmahoYeh:gitter.im` (only `m.room.create`)
- `!OpjflmXuauidodzmJH:gitter.im` (only `m.room.create`)
- `!QCVddUkjrHLjEqeXQd:gitter.im` (no events)
- `!wJBkwhqwLssHVjKErI:gitter.im` (only `m.room.create`)
- `!kkjacXiigblOffltrw:gitter.im` (only `m.room.create`)
- `!etJKlYVRXEtkkMdEOJ:gitter.im` (no events)
- `!VeVZEnGasXJXpAYECh:gitter.im` (only `m.room.create`)
- `!iJIbqcgSzsAFnwjVBI:gitter.im` (no events)
- `!koFtykttKGRpjbzflq:gitter.im` (no events)
- `!mxfIojpZcyQDBbmmoY:gitter.im` (only `m.room.create`)
- `!ezxePTXYHMSIkpGOiy:gitter.im` (only `m.room.create`)
- `!QlEqqNHuTSIHswllOX:gitter.im` (only `m.room.create`)
- `!XInKHVcDAmmyGBEUwr:gitter.im` (no events)
- `!KIKPFcoDnhBeZhLNUr:gitter.im` (only `m.room.create`)
- `!BcBAOZpxjTAvMDfSHf:gitter.im` (only `m.room.create`)
- `!DuyTBdNdlhfXoBFNqr:gitter.im` (only `m.room.create`)
- `!qgfkmcpByUudrFHTMX:gitter.im` (no events)
- `!tFwVYIoUNZQhRhjFNO:gitter.im` (no events)
- `!qoZZIEjvblGKIkqEdj:gitter.im` (no events)

---

In terms of moving Gitter forward, I think we can safely reconcile this by removing the bridged room entry in the Gitter database and the Matrix room alias and have them re-created (tracked by https://gitlab.com/gitterHQ/webapp/-/issues/2858#note_1267698515). But I'm unable to remove the Matrix room alias from the old room (`M_FORBIDDEN`) 😬

And it would still be good to fix the Synapse bugs that allow this to happen.

### Steps to reproduce

1. Not sure how this happened but it's all Gitter appservice interactions against Synapse

### Homeserver

gitter.im

### Synapse Version

1.74.0

### Installation Method

I don't know

### Database

PostgreSQL

### Workers

Multiple workers

### Platform

`gitter.im` (`gitter.ems.host`) runs on EMS but I don't know the details

Python version: `3.11.1`

### Configuration

_No response_

### Relevant log output

The only relevant logs for this room from the time of creation is this message:

https://modular-euwest2-kibana.proxy.matrix.org/app/kibana#/discover?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-8w,to:now-3w))&_a=(columns:!(_source),index:'67745970-aa2d-11ea-96a7-efc84863d0ea',interval:auto,query:(language:kuery,query:'kubernetes.pod.labels.hostname:%22gitter.ems.host%22%20AND%20message:%22!!gHbYoGxiEuXmahoYeh:gitter.im%22'),sort:!(!('@timestamp',desc)))

```
2023-01-09 17:33:53,203 - synapse.replication.http.send_events - 149 - INFO - POST-4249- Got batch of events to send, last ID of batch is: $JeDmcg6bYb0s-d6pxIbgmfPaJoChH-qWlnTZ3n8xGdU, sending into room: !gHbYoGxiEuXmahoYeh:gitter.im
```

---

Database information:

```
# Connect to the database: https://gitlab.matrix.org/new-vector/internal/-/wikis/EMS-postgres#connecting-to-an-ems-database
$ ems-db-debug gitter.im

select * FROM rooms WHERE room_id = '!gHbYoGxiEuXmahoYeh:gitter.im';
room_id | is_public | creator | room_version | has_auth_chain_index
-------------------------------+-----------+--------------------------+--------------+----------------------
!gHbYoGxiEuXmahoYeh:gitter.im | t | @gitter-badger:gitter.im | 9 | t

select * FROM room_memberships WHERE room_id = '!gHbYoGxiEuXmahoYeh:gitter.im';
event_id | user_id | sender | room_id | membership | forgotten | display_name | avatar_url
----------+---------+--------+---------+------------+-----------+--------------+------------
(0 rows)

select * FROM events WHERE room_id = '!gHbYoGxiEuXmahoYeh:gitter.im';
topological_ordering | event_id | type | room_id | content | unrecognized_keys | processed | outlier | depth | origin_server_ts | received_ts |
sender | contains_url | instance_name | stream_ordering | state_key | rejection_reason
----------------------+----------------------------------------------+---------------+-------------------------------+---------+-------------------+-----------+---------+-------+------------------+---------------+-------
-------------------+--------------+---------------------------+-----------------+-----------+------------------
1 | $JeDmcg6bYb0s-d6pxIbgmfPaJoChH-qWlnTZ3n8xGdU | m.room.create | !gHbYoGxiEuXmahoYeh:gitter.im | | | t | f | 1 | 1673285632772 | 1673285633224 | @gitte
r-badger:gitter.im | f | synapse-event-persister-3 | 94262741 | |
(1 row)

select * FROM room_aliases WHERE room_id = '!gHbYoGxiEuXmahoYeh:gitter.im';
room_alias | room_id | creator
----------------------------------+-------------------------------+--------------------------
#GreenShoeGarage_Lobby:gitter.im | !gHbYoGxiEuXmahoYeh:gitter.im | @gitter-badger:gitter.im
(1 row)
```

### Anything else that would be useful to know?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.