Optimize /keys/query endpoint

This issue has been migrated from [#14706](https://github.com/matrix-org/synapse/issues/14706).

---

This endpoint seems to do some funny things (and the code is hard to follow, but that doesn't seem to be a performance issue). Note that this endpoint is supported by workers (and goes the `client_reader` on matrix.org/Complement). See `E2eKeysHandler.query_devices` for the entry point.

* It linearizes based on requesting user / device (but since you can run multiple of these, I'm not sure how useful that is).
* It seems to reasonably separate queries into local and remote users, checking caches and such.
* For any uncached remote queries it fans-out per destination (via background tasks) and calls `E2eKeysHandler._query_devices_for_destination`.
* This then iterates through each user for that destination and attempts to resync devices via the `/user/devices/{user_id}` federation endpoint.
* If workers are used, this calls a replication endpoint on the main process.
* In some situations this might fail and Synapse then tries calling `/user/keys/query` once per destination.

Note that Synapse will call the same resync logic for a remove users keys/devices due to a few different operations (receiving to-device messages from an unknown device, a `/keys/query` request in some situations, maybe other situations).

----

* Can we avoid bouncing this to the main process? (Can we have a different worker handle these queries?)
* Can we shard this work? Likely by destination. (Note the `/user/keys/query` is *not* sent to the main process.)
* Can we parallelize the per-user queries (or add a new endpoint to make them more efficient)?