GET /members is slow, taking >30s to respond for Matrix HQ

### Description

Element X cannot load the member list because the request times out. Curling the endpoint shows it takes over 30s, in some cases over a minute (on conduit.rs: 23s cold or 4s when cached).

Maybe clients should use /joined_members instead? (Only takes ~2s on matrix.org, 0.7s on conduit.rs)

### Steps to reproduce

curl /members for !OGEhHVWSdvArJzumhm:matrix.org

### Homeserver

matrix.org

### Synapse Version

1.100.0rc1 (b=matrix-org-hotfixes,d5e213e16f)

### Installation Method

I don't know

### Database

-

### Workers

I don't know

### Platform

-

### Configuration

_No response_

### Relevant log output

```shell
-
```

### Anything else that would be useful to know?

_No response_