element-hq / element-hq/synapse

[metrics] Background process 'delayed_events.notify_new_event' threw an exception

Open
#18,351 1 comment 3 reactions 0 assignees View on GitHub
O-Frequent S-Tolerable T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

### Description

Metrics keeps throwing errors from delayed_events for rejected events
`synapse.api.errors.NotFoundError: 404: Could not find event $ByeSbu0iUhv0RB4MSH047mwIObVRuHLYspnLZDjklOk`

From `events` table:
|topological_ordering|event_id|type|room_id|content|unrecognized_keys|processed|outlier|depth|origin_server_ts|received_ts|sender|contains_url|instance_name|stream_ordering|state_key|rejection_reason|
|--------------------|--------|----|-------|-------|-----------------|---------|-------|-----|----------------|-----------|------|------------|-------------|---------------|---------|----------------|
|17782|$ByeSbu0iUhv0RB4MSH047mwIObVRuHLYspnLZDjklOk|m.room.member|!NasysSDfxKxZBzJJoE:matrix.org|||true|true|17782|1611758224059|1745096815930|@yan:yetanothernerd.xyz|false|event_stream_writer-0|-43538330|@yan:yetanothernerd.xyz|auth_error|

### Steps to reproduce

- join !NasysSDfxKxZBzJJoE:matrix.org?via=matrix.org

### Homeserver

rory.gay

### Synapse Version

1.127.1

### Installation Method

Other (please mention below)

### Database

single postgres, no port/restore

### Workers

Multiple workers

### Platform

nixos/unstable, dedicated host

### Configuration

presence enabled
experimental_features = {
"msc2815_enabled" = true; # Redacted event content
"msc3026_enabled" = true; # Busy presence
"msc3266_enabled" = true; # Room summary API
"msc3916_authenticated_media_enabled" = true; # Authenticated media
"msc3823_account_suspension" = true; # Account suspension
"msc4151_enabled" = true; # Report room API (CS-API)
};
https://cgit.rory.gay/Rory-Open-Architecture.git/tree/host/Rory-ovh/services/matrix/synapse/synapse-main.nix

### Relevant log output

```shell
Apr 19 22:09:01 Rory-nginx synapse[2570433]: synapse.metrics.background_process_metrics: [delayed_events.notify_new_event-7264596] Background process 'delayed_events.notify_new_event' threw an exception
Traceback (most recent call last):
File "/nix/store/ibc5r4abj06040pjsaa8qlc8s9jl1v00-matrix-synapse-1.127.1/lib/python3.12/site-packages/synapse/metrics/background_process_metrics.py", line 251, in run
return await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/ibc5r4abj06040pjsaa8qlc8s9jl1v00-matrix-synapse-1.127.1/lib/python3.12/site-packages/synapse/handlers/delayed_events.py", line 138, in process
await self._unsafe_process_new_event()
File "/nix/store/ibc5r4abj06040pjsaa8qlc8s9jl1v00-matrix-synapse-1.127.1/lib/python3.12/site-packages/synapse/handlers/delayed_events.py", line 183, in _unsafe_process_new_event
await self._handle_state_deltas(deltas)
File "/nix/store/ibc5r4abj06040pjsaa8qlc8s9jl1v00-matrix-synapse-1.127.1/lib/python3.12/site-packages/synapse/handlers/delayed_events.py", line 210, in _handle_state_deltas
event = await self._store.get_event(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/ibc5r4abj06040pjsaa8qlc8s9jl1v00-matrix-synapse-1.127.1/lib/python3.12/site-packages/synapse/storage/databases/main/events_worker.py", line 548, in get_event
raise NotFoundError("Could not find event %s" % (event_id,))
synapse.api.errors.NotFoundError: 404: Could not find event $ByeSbu0iUhv0RB4MSH047mwIObVRuHLYspnLZDjklOk
```

### Anything else that would be useful to know?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.