[tracker] `pyupgrade --py36-plus`

This issue has been migrated from [#9744](https://github.com/matrix-org/synapse/issues/9744).

---

This is a tracker issue for running and PR-ing automated refactors using [pyupgrade](https://github.com/asottile/pyupgrade), per #9131.

I'll do this in the following changes:
- [x] Remove all `# -*- coding: utf-8 -*-` lines (#9786)
- [ ] `pyupgrade` the codebase with `--keep-precent-format`
- [x] `tests/` (#10347)
- [x] `synapse/` (#10348)
- [ ] Other (#XXXX)
- [ ] `pyupgrade` without `--keep-precent-format` (#XXXX)

### Overall Extra Info

Some common changes from `pyupgrade`:
- Changes `%`-formatting to `.format` (or f-strings if non-trivial)
- Removes `coding: utf-8` file header (not needed since python 3)
- Changes `typing.TypedDict("A")` class instantiations to `class A(TypedDict)`