element-hq / element-hq/synapse

Unhandled IndexError in guest-macaroon auth path returns 500 instead of 401 for tokens that base64-decode to empty

Open Beginner friendly
#20,188 0 comments 0 reactions 0 assignees View on GitHub
A-Validation T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

### Description

Any `access_token` whose lenient base64 decode yields zero bytes reaches the guest-macaroon fallback in `get_user_by_access_token()`. `Macaroon.deserialize` on empty bytes then raises `IndexError` inside pymacaroons, which is not in the caught exception tuple — so the request fails with HTTP 500 `M_UNKNOWN` instead of a clean 401, and a full traceback (~39 log lines vs 3 for a normal 401) is written per request.

### Affected versions

- `matrixdotorg/synapse:latest` = 1.160.0 (reproduced 2026-09-05, Docker, fresh generated config)
- Also reproduced on 1.159.0
- `develop` as of 2026-09-05: `synapse/api/auth/internal.py` still catches only `MacaroonException, TypeError, ValueError` — unfixed

### Reproduction

Start a stock Synapse (docker guide default) and send a request with a token that decodes to empty bytes:

curl -sk -w '\nHTTP %{http_code}\n' \
'http://localhost:8008/_matrix/client/v3/account/whoami?access_token=*'

Buggy result: HTTP 500, body `{"errcode":"M_UNKNOWN","error":"Internal server error"}`

Controls — `zzz` and empty are handled correctly, empty-decoding payloads are not:

- token `*` → HTTP 500 `M_UNKNOWN`
- token `!` → HTTP 500 `M_UNKNOWN`
- token `~~~` → HTTP 500 `M_UNKNOWN`
- token `zzz` → HTTP 401 `M_UNKNOWN_TOKEN` (correct)
- token empty → HTTP 401 `M_UNKNOWN_TOKEN` (correct)

Any payload whose lenient base64 decode results in zero bytes triggers it (`*`, `!`, `~~~`, spaces). Payloads containing `-` or `_` (valid urlsafe-alphabet chars) decode to non-empty bytes and correctly return 401.

The bug is not limited to the query parameter — the `Authorization` header path and other authenticated endpoints behave identically:

curl -sk -o /dev/null -w '%{http_code}\n' \
-H 'Authorization: Bearer ***' \
'http://localhost:8008/_matrix/client/v3/account/whoami'

curl -sk -o /dev/null -w '%{http_code}\n' \
'http://localhost:8008/_matrix/client/v3/sync?access_token=*'

curl -sk -o /dev/null -w '%{http_code}\n' -X POST \
'http://localhost:8008/_matrix/client/v3/rooms/!test:example.com/kick?access_token=*'

All three return HTTP 500.

### Traceback (from server log)

Traceback (most recent call last):
File ".../synapse/http/server.py", line 336, in _async_render_wrapper
callback_return = await self._async_render(request)
File ".../synapse/rest/client/account.py", line 869, in on_GET
requester = await self.auth.get_user_by_req(request, allow_guest=True)
File ".../synapse/api/auth/internal.py", line 250, in get_user_by_access_token
user_id = self._macaroon_generator.verify_guest_token(token)
...
File ".../pymacaroons/macaroon.py", line 39, in deserialize
return serializer.deserialize(serialized)
File ".../pymacaroons/serializers/binary_serializer.py", line 96, in deserialize_raw
first = six.byte2int(serialized[:1])
IndexError: index out of range

### Root cause

`base64.b64decode` in non-strict mode strips characters outside the alphabet, so `*` decodes to empty bytes. `binary_serializer.py:96` then indexes into empty bytes (`serialized[:1]` is empty, `six.byte2int` raises `IndexError`). The except clause in `synapse/api/auth/internal.py` only covers:

except (
pymacaroons.exceptions.MacaroonException,
TypeError,
ValueError,
) as e:

`IndexError` slips through to the generic 500 handler.

### Suggested fix

Add `IndexError` to the caught exceptions in `get_user_by_access_token()` (or validate that the decoded payload is non-empty before calling `deserialize()`). Longer term, a guard in pymacaroons' `deserialize_raw` would fix the class of issue for all consumers.

### Impact

No security impact demonstrated: the server does not crash (verified healthy after 50 consecutive payloads — `/login` still returns 200), no data is disclosed to the client (traceback is server-log only), and authentication still denies access. The practical effects are the wrong status code and per-request traceback log amplification (39 lines vs 3).

Originally reported to security@element.io, closed as "unable to identify the security implications" — filing here as a code defect / error-handling bug.

Reported-by: KITLAB13

Contributor guide

Open the contributing guide

Research direction

Start in synapse/api/auth/internal.py at get_user_by_access_token(), then review the listed whoami, sync, and kick curl reproductions. The work is complete when empty-decoding tokens consistently produce HTTP 401 M_UNKNOWN_TOKEN without an unhandled traceback, while normal valid and invalid token cases remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
authentication, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.