element-hq / element-hq/synapse

Long running user deactivations via admin API are not deduplicated and can starve DB connections

Open
#16,055 0 comments 0 reactions 0 assignees View on GitHub
A-Account-Deactivation A-Admin-API S-Minor T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#16055](https://github.com/matrix-org/synapse/issues/16055).

---

https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#deactivate-account can be used to deactivate a users session. However, because it does (potentially) expensive things like deleting devices it can take a long time to run. For serious device hoarders, this can cause the request to time out.

The HTTP request will time out but the DB query does not, leading to the case where it's an easy footgun to retry a "failed" user deactivation and end up having multiple concurreent slow queries jamming up the database connection pool. Do it enough times, or to enough users and the main process will be starved out of connections.

Separately, it's probably not good that deactivations are taking minutes to run.

Example output:

SELECT left(query, 90) as query, state FROM pg_stat_activity WHERE state IS NOT NULL AND query LIKE 'DELETE%';
```
query | state
--------------------------------------------------------------------------------------------+--------
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM device_inbox WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3 | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM device_inbox WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9 | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM device_inbox WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6 | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE1','DEVICE2','DEVICE3','YI | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE44','DEVICE5','DEVICE6','IA | active
DELETE FROM devices WHERE device_id = ANY(ARRAY['DEVICE7','DEVICE8','DEVICE9','OD | active

```

Contributor guide

Open the contributing guide

Research direction

Start with the admin API's deactivate-account endpoint described in the linked documentation, then reproduce the concurrent requests while observing the provided pg_stat_activity query. Done means repeated deactivation requests no longer create duplicate long-running database work or starve the connection pool; the issue also notes that deactivations taking minutes may need separate investigation.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
api, backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.