Actyx / Actyx/Actyx

WIP: versioning of events

Offen
#45 10 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Actyx Feature
Vorherrschende Sprache
Rust
Sterne
294
Forks
11
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

To make sure that nulling out events remains a feature of last resort, we need to, in the medium term, offer a better alternative. I keep coming back to schema migration as a generic solution for many such issues.

E.g. you give the admin a way to control what events the app sees, based on some additional tags which are configured by the admin instead of inside the app. This *must* be in the hands of the admin, since the developer might not be available, and certainly does not enjoy being called for an emergency on Friday night.

Then rewrite the offending events into a new stream (in the easiest case, skip an offending event), add the new tag, and redeploy the app. The old events are still there, so existing, not yet reconfigured versions of the app will be unaffected. But the newly deployed app - which is completely unchanged in terms of code, will only see the new, rewritten events and only emit events that are marked with the new, admin-configured tag.

What this is about is a form of versioning. But it is not versioning of the app itself, which is done by the app developer, but versioning of the emitted and consumed events.

Here is a proposal. When an app is deployed, all events by the app are augmented with an event version tag. So e.g. you deploy com.actyx.mwl, you as an admin can assign a version number for the *events*.

Then, to solve a problem of broken events, you can do the following:

- increase the version number of the events emitted and seen by the app
- convert all existing events from `e-v1` to `e-v2`. This could be a batch job, assuming that the v1 app is silent, or a running query, assuming the `v1` app is still active. This conversion would be something like a tiny program configured via an AQL query that again emits events
- redeploy/reassign the app to `e-v2`. It would see just the converted events from `e-v1`.
- as soon as it is certain that there is no `e-v1` app remaining in the swarm, the converter can be shut off. The `e-v1` events will still be in the system for historical purposes. Nothing has been deleted.

This is a kinda brute force way to do this. In the future we might want to do this in a more fine-grained way, by versioning based on tags. I don't know yet.

An analogy would be what we used to do in the past, e.g. for CTA, where we changed the topic and then injected the relevant data from the previous topic.

By using app ids this is more fine grained, and by using aql queries for producing event logs, this is more approachable for SIs.

The first application of this would be to get rid of offending events, or to do small changes in events like fixing a typo in a property. But this can evolve into a genuine, albeit not yet very elegant, schema migration system.

E.g. you have v2 of mwl which has an improved event model. Instead of having lots of complexity in your code dealing with the legacy events, or having very strict limitations in how you can change your event model, you just have a standalone tool that converts events from `a-v1` to `a-v2`, which runs on some node until v2 has been deployed everywhere. After the conversion, the logic of `a-v2` can be completely unburdened by the past.

Note: this might be a longer term solution for the problem described in https://github.com/Actyx/Cosmos/issues/7169 , and other problems that will come up as you evolve an app and its event model.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.