abcxyz / abcxyz/github-metrics-aggregator

Leech Pipeline - FAILURE status handling

Offen
#58 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
enhancement
Vorherrschende Sprache
Go
Sterne
16
Forks
3
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

### TL;DR

Logs with a FAILURE status are currently filtered out of the query.

We need to figure out a way to mark how many attempts have been made for a particular delivery id preferably without causing an update to the row in BigQuery.

The simplest approach would be to just not write a FAILURE status and let the query try again.

The drawback is that something could be stuck forever in that situation and we wouldn't want to keep processing it. I think a secondary FAILURE table might work that we can join into the main query as WHERE count(failures where delivery_id = x) < 10 or something.

This adds complexity to the write operation though so it requires some thought.

### Detailed design

_No response_

### Alternatives considered

_No response_

### Additional information

_No response_

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.