element-hq / element-hq/synapse

CI trial runs annotated with `Signatures do not match`

Open
#10,942 0 comments 0 reactions 0 assignees View on GitHub
P4 S-Tolerable T-Task
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#10942](https://github.com/matrix-org/synapse/issues/10942).

---

1. GHA invokes trial which runs the test `tests.handlers.test_oidc.OidcHandlerTestCase.test_callback_session`.
2. This encounters [debug logging](https://github.com/matrix-org/synapse/blob/47854c71e9bded2c446a251f3ef16f4d5da96ebe/synapse/handlers/oidc.py#L215-L227) added in #9307. This uses `logger.exception` which writes a traceback to synapse's log. It will eventually appear in the test output when trial summaries the test run. (Unclear if this is on stdout or stderr)
3. We use actions/setup-python which [sets up a problem matcher](https://github.com/actions/setup-python/blob/2d803e7feaf26323835f529f28efa6400f18ad01/.github/python.json#L12) that looks for tracebacks ending in a `raise` statement containing a string literal.
4. The (intentionally-logged) tracebacks end up getting flagged as a scary annotation. But this is a false positive.

![image](https://user-images.githubusercontent.com/8614563/135282508-08f66760-935e-413b-aa34-50b646cc77bf.png)

Suggestions:

- Remove the `logger.exception` if we don't need it any more
- Log something else that's useful but which won't trigger the problem matcher
- Setup test logging config to suppress logging.exceptions in this module

(Surprised this hasn't caught annotated more exceptions in the logs tbh---perhaps [the regex](https://github.com/actions/setup-python/blob/2d803e7feaf26323835f529f28efa6400f18ad01/.github/python.json#L12) only detects single-quoted string literal exception messages?)

Contributor guide

Open the contributing guide

Research direction

Start with synapse/handlers/oidc.py at the debug logging linked in the issue and reproduce tests.handlers.test_oidc.OidcHandlerTestCase.test_callback_session under the GitHub Actions setup-python problem matcher. Review the listed logging or test-configuration options, then verify that the trial output no longer produces a false “Signatures do not match” annotation while retaining useful test diagnostics.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.