clearlydefined / clearlydefined/service
Add support for data change notifications
- Dominant language
- TypeScript
- Stars
- 51
- Forks
- 46
- PR merge metrics
- No merged PRs in 30d
Description
The need to enable ClearlyDefined data replication has been discussed already (#386). I'd like to revive this discussion and propose to approach this problem as a 2 step process:
1. Enable an externally visible notifications about changes in definitions, from both harvesting and curation. An event stream, if you'd like.
2. Implement data dump of all the existing definitions, with a selected set of attributes, to an external location (e.g. a file, a storage bucket, etc.).
This issue focuses on the first part, enabling the data change notifications. I propose to use a persistent web hook mechanism for this purpose. The list of requirements for ClearlyDefined service to implement this looks roughly like the following:
- Call an external web service when some data change happens.
- This call should keep on working after ClearlyDefined service restarts without any actions from the recipient (the persistent part of the definition above).
- Call URL should support secrets, e.g. auth credentials, without hardcoding them in the ClearlyDefined source code (obviously).
- There should be support for multiple web hooks, to have multiple events destinations.
- It is ok to use an ad-hoc, low-level approach for setting up new web hooks, e.g. via a PR to ClearlyDefined service. No web UI is required, at least initially.
Please feel free to add further requirements if they seem necessary, but having this seems to be sufficient for a start.
Contributor guide
Assessment
This issue has not been assessed yet.