element-hq / element-hq/synapse
Remove the concept of worker "types"
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
This issue has been migrated from [#10441](https://github.com/matrix-org/synapse/issues/10441).
---
We're currently midway through the process of fading out the concept of different types for a Synapse worker process. For example `synapse.app.pusher`, `synapse.app.federation_sender` etc. While initially this provided a quick way to configure a worker to perform certain actions, it is also rather confusing and the inability to specify multiple types per worker can limit the flexibility of deployment.
For instance, it's currently not possible to configure a worker to both send events to appservices (`synapse.app.appservice`) and update the user directory tables (`synapse.app.user_dir`), as the functionality is forcefully disabled unless the worker is specifically marked as each respective type:
https://github.com/matrix-org/synapse/blob/85d237eba789a667109ced140026d2494b210310/synapse/app/generic_worker.py#L433-L463
Instead, workers should be configured via worker file config options, so that one could simply set `notify_appservices: true` and `update_user_directory: true` in the worker config to enable the worker to handle both sets of tasks.
As many worker deployments currently rely on the `worker_type` configuration, a deprecation period is necessary. A rough plan for carrying this out could look like:
* Allow all current worker configurations through config file values alone.
* Announce a deprecation period for the `worker_type` config option.
* After some time, remove the `worker_type` config option. All workers are now a "generic worker", though this term is no longer a necessary distinction.
In the short term, the first step would allow for more immediate flexibility for medium-sized deployments - which need to move tasks off the main process, but don't have the resources to spin up a separate worker for each type.
## Superseded worker types so far
- [x] `synapse.app.appservice` #12452
- [x] `synapse.app.user_dir` #12654
- [ ] `synapse.app.pusher`
- [ ] `synapse.app.federation_sender`
- [ ] `synapse.app.media_repository`
- [ ] `synapse.app.frontend_proxy`
Contributor guide
Assessment
This issue has not been assessed yet.