clearlydefined / clearlydefined/service

Add support for data change notifications

Open
#958 35 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
51
Forks
46
PR merge metrics
No merged PRs in 30d

Description

The need to enable ClearlyDefined data replication has been discussed already (#386). I'd like to revive this discussion and propose to approach this problem as a 2 step process:

1. Enable an externally visible notifications about changes in definitions, from both harvesting and curation. An event stream, if you'd like.
2. Implement data dump of all the existing definitions, with a selected set of attributes, to an external location (e.g. a file, a storage bucket, etc.).

This issue focuses on the first part, enabling the data change notifications. I propose to use a persistent web hook mechanism for this purpose. The list of requirements for ClearlyDefined service to implement this looks roughly like the following:

- Call an external web service when some data change happens.
- This call should keep on working after ClearlyDefined service restarts without any actions from the recipient (the persistent part of the definition above).
- Call URL should support secrets, e.g. auth credentials, without hardcoding them in the ClearlyDefined source code (obviously).
- There should be support for multiple web hooks, to have multiple events destinations.
- It is ok to use an ad-hoc, low-level approach for setting up new web hooks, e.g. via a PR to ClearlyDefined service. No web UI is required, at least initially.

Please feel free to add further requirements if they seem necessary, but having this seems to be sufficient for a start.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.