element-hq / element-hq/synapse

API endpoint `/_synapse/admin/v1/statistics/database/rooms` throwing Python error

Open
#15,821 0 comments 0 reactions 0 assignees View on GitHub
A-Admin-API O-Uncommon S-Minor T-Defect
Dominant language
Python
Stars
4.6k
Forks
607
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#15821](https://github.com/matrix-org/synapse/issues/15821).

---

### Description

When attempting to use the API endpoint `/_synapse/admin/v1/statistics/database/rooms` via `curl`, a HTTP 500 is received while Synapse throws Python errors in the log.

### Steps to reproduce

1. `curl -XGET --header "Authorization: Bearer TOKENTOKEN" http://synapse-server/_synapse/admin/v1/statistics/database/rooms`

### Homeserver

`aguiarvieira.pt`

### Synapse Version

1.86.0

### Installation Method

Docker (`matrixdotorg/synapse`)

### Database

PostgreSQL

### Workers

Multiple workers

### Platform

Oracle Arm Free with Docker

### Configuration

Presence: yes
Retention: no
Synapse modules: shared_secret_authenticator.SharedSecretAuthProvider

### Relevant log output

```shell
matrix-matrix-synapse-1 | 2023-06-23 10:33:19,794 - synapse.http.server - 133 - ERROR - GET-3694 - Failed handle request via 'LargestRoomsStatistics':
matrix-matrix-synapse-1 | Traceback (most recent call last):
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/http/server.py", line 319, in _async_render_wrapper
matrix-matrix-synapse-1 | callback_return = await self._async_render(request)
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/http/server.py", line 529, in _async_render
matrix-matrix-synapse-1 | callback_return = await raw_callback_return
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/rest/admin/statistics.py", line 133, in on_GET
matrix-matrix-synapse-1 | room_sizes = await self.stats_controller.get_room_db_size_estimate()
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/storage/controllers/stats.py", line 93, in get_room_db_size_estimate
matrix-matrix-synapse-1 | await self.stores.main.db_pool.runInteraction(
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/storage/database.py", line 930, in runInteraction
matrix-matrix-synapse-1 | return await delay_cancellation(_runInteraction())
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks
matrix-matrix-synapse-1 | result = context.run(
matrix-matrix-synapse-1 | ^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator
matrix-matrix-synapse-1 | return g.throw(self.type, self.value, self.tb)
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/storage/database.py", line 896, in _runInteraction
matrix-matrix-synapse-1 | result = await self.runWithConnection(
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/storage/database.py", line 1025, in runWithConnection
matrix-matrix-synapse-1 | return await make_deferred_yieldable(
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/twisted/python/threadpool.py", line 244, in inContext
matrix-matrix-synapse-1 | result = inContext.theWork() # type: ignore[attr-defined]
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/twisted/python/threadpool.py", line 260, in
matrix-matrix-synapse-1 | inContext.theWork = lambda: context.call( # type: ignore[attr-defined]
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/twisted/python/context.py", line 117, in callWithContext
matrix-matrix-synapse-1 | return self.currentContext().callWithContext(ctx, func, *args, **kw)
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/twisted/python/context.py", line 82, in callWithContext
matrix-matrix-synapse-1 | return func(*args, **kw)
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
matrix-matrix-synapse-1 | result = func(conn, *args, **kw)
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/storage/database.py", line 1018, in inner_func
matrix-matrix-synapse-1 | return func(db_conn, *args, **kwargs)
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/storage/database.py", line 758, in new_transaction
matrix-matrix-synapse-1 | r = func(cursor, *args, **kwargs)
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | File "/usr/local/lib/python3.11/site-packages/synapse/storage/controllers/stats.py", line 90, in get_room_db_size_estimate_txn
matrix-matrix-synapse-1 | for room_id, freq in zip(vals, freqs):
matrix-matrix-synapse-1 | ^^^^^^^^^^^^^^^^
matrix-matrix-synapse-1 | TypeError: 'NoneType' object is not iterable
```

### Anything else that would be useful to know?

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the curl request, then inspect synapse/rest/admin/statistics.py and synapse/storage/controllers/stats.py around get_room_db_size_estimate_txn, where the traceback shows a NoneType iteration. Trace why the database statistics query produces a missing value. Done means the endpoint no longer returns HTTP 500 and returns room statistics successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, postgresql, python
Domain
api, backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.