element-hq / element-hq/synapse
Unhandled ijson exceptions in various places causing log and sentry spam
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
This issue has been migrated from [#14320](https://github.com/matrix-org/synapse/issues/14320).
---
https://sentry.tools.element.io/organizations/element/issues/29/?project=2&query=is%3Aunresolved&statsPeriod=24h
```
2022-10-28 02:43:48,424 - twisted - 274 - ERROR - _maybe_retry_device_resync-852- - Exception ignored in:
2022-10-28 02:43:48,430 - twisted - 274 - ERROR - _maybe_retry_device_resync-852- - Traceback (most recent call last):
2022-10-28 02:43:48,435 - twisted - 274 - ERROR - _maybe_retry_device_resync-852- - File "/home/synapse/env-pyston-poetry/lib/python3.8-pyston2.3/site-packages/ijson/backends/python.py", line 46, in utf8_encoder
2022-10-28 02:43:48,440 - twisted - 274 - ERROR - _maybe_retry_device_resync-852- - target.close()
2022-10-28 02:43:48,443 - twisted - 274 - ERROR - _maybe_retry_device_resync-852- - File "/home/synapse/env-pyston-poetry/lib/python3.8-pyston2.3/site-packages/ijson/backends/python.py", line 116, in Lexer
2022-10-28 02:43:48,447 - twisted - 274 - ERROR - _maybe_retry_device_resync-852- - target.send(EOF)
2022-10-28 02:43:48,450 - twisted - 274 - ERROR - _maybe_retry_device_resync-852- - File "/home/synapse/env-pyston-poetry/lib/python3.8-pyston2.3/site-packages/ijson/backends/python.py", line 161, in parse_value
2022-10-28 02:43:48,453 - twisted - 274 - ERROR - _maybe_retry_device_resync-852- - raise common.IncompleteJSONError('Incomplete JSON content')
2022-10-28 02:43:48,456 - twisted - 274 - ERROR - _maybe_retry_device_resync-852- - ijson.common.IncompleteJSONError: Incomplete JSON content
```
Contributor guide
Research direction
Start with the _maybe_retry_device_resync path named in the log and trace where ijson parses the response. Reproduce or inspect the IncompleteJSONError handling at that call site and identify the other affected locations; done means expected parsing failures no longer generate repeated log or Sentry noise while genuine failures remain visible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100