matrix-org / matrix-org/sytest
tests/50federation/10query-profile.pl flaky "Gateway Time-out"
Nobody has claimed this yet.
- Dominant language
- Perl
- Stars
- 81
- Forks
- 63
- Avg merge
- 1h 47m
- Merged PRs (30d)
- 1
Description
10query-profile.pl seems to be somewhat unreliable, sometimes fails with
```
Running tests/50federation/10query-profile.pl...
Testing if: Outbound federation can query profile data... FAIL:
| HTTP Request failed (504 Gateway Time-out) at ./run-tests.pl line 542.
+----------------------
```
Moreover, if it fails in this manner it fails *immediately* claiming that timeout, rather than starting up, waiting for a bit, then times out.
Further investigation shows that if you run that one test on its own it seems to quite reliably pass:
```
$ ./run-tests-in-tox --dendron=../dendron/bin/dendron --haproxy tests/50federation/10query-profile.pl
Running tests/50federation/10query-profile.pl...
...
All tests PASSED
```
Moreover, various combinations of tests before it seem to indicate that it is server-startup timing related. By including or omitting various other parts of the test suite before it, if it manages to get to this test file within 1m40 of starting sytest, this test script always passes. If however it hasn't got there before at least 2m06, then this test script always fails.
This also might suggest why this test *seems* to be dendron/haproxy-related, at least on my laptop, because it has always taken longer than this time limit to get there (and hence fails) if running via haproxy+postgres, but gets there much sooner (and hence passes) if running via in-memory sqlite.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/50federation/10query-profile.pl and run it through ./run-tests-in-tox using the dendron and haproxy setup; compare isolated and full-suite runs. Trace startup timing around the 1m40/2m06 threshold and the 504 reported by run-tests.pl line 542. Done means the test no longer flakes in the affected suite while retaining its existing pass behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- perl, postgresql, sqlite
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100