Expose user_directory to the module system

This issue has been migrated from [#12247](https://github.com/matrix-org/synapse/issues/12247).

---

**Description:**

It would be nice if modules could extend the user_directory search. If I am right, there is no callback interface, and you cannot register the needed web resource in a module, by the current restrictions
Reasons/example scenario:
- the [ldap auth backend](https://github.com/matrix-org/matrix-synapse-ldap3) handles the job of relying on a central, non-synapse managed user database. Other backends also apply
- if an auth backend is used, you either:
- can't find the existing (on the backend) users, or
- with config `user_directory.seach_all_users: true` you find **all** (maybe? not tested). I tested without an auth backend, and with this setting, I find bridge puppet bots, I think even ones not relevant for me (created for another user)
- there should be something in-between the previous two options, like permit to find anybody in the same LDAP organization unit or group. In other words, the discoverability could (or rather should, in some deployments) be sourced from the same backend the auth one uses, then merged with synapse's user search result
- ma1sd kinda did this, but that is abandonware (and the whole identity server thing needs love), and it made it work by intercepting the `/_matrix/client/r0/user_directory/` HTTP requests, which is not elegant at all. The module system would be the optimal place to do this.

Callbacks would be better than selectively enabling the mentioned resource, I presume.