algorand / algorand/conduit

Graceful Pipeline Exit

Ouverte
#100 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
Team Lamprey
Langage dominant
Go
Étoiles
44
Forks
34
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

## Problem

Currently, when the main conduit process receives a shutdown signal, it abruptly tears down all its goroutines even though the pipeline shares a common `context.Context` object that is supposed to make it possible to exit more gracefully and in a deterministic fashion. In particular, it is currently not possible to coordinate a new requirement such as having each plugin finish the current round before exit.

## Solution

This story posits the following goals (though after further discussion we may want to trim the goals or break them out into separate issues):

1. Handle an interrupt signal in a way that won't cause duplicate calls to plugin `Stop()`. As a starting point refer to #97 but better crafting of where to place the signal handling is required. See also #99
2. Consider adding a new `conduit stop` command that looks for the PID file and sends an interrupt to that process
3. Consider the possibility of letting each plugin go to the end. Even better make this configurable via the [type of interrupt signal received](https://cs.opensource.google/go/go/+/refs/tags/go1.20.5:src/syscall/zerrors_linux_amd64.go;l=1321). EG:
* `conduit stop now` would send an `os.Interrupt` (or a `kill -TERM $(PROCNUM)` ) and kills all plugin processes immediately
* `conduit stop end-of-round` produces something like `kill -USR1 $(PROCNUM)` and kills plugin process in the following order at the end of its current round: `Importer` → `Processors` → `Exporter`

### Some questions

1. Should we add `WhyStopped()` to the Pipeline interface, and send a telemetry observation with this context cancellation cause?
2. Do we still need `Stop()` in the Pipeline's interface?
3. Do we need sentinel error to act as sentinel cancelation causes and to distinguish between different legitimate reasons?

####
Some possible code in the client making use of the stop cause:

```go
defer func() {
pline.Stop()
if !errors.Is(pline.WhyStopped(), pipeline.BecauseStopMethod) {
logger.Error("unexpected conduit pipeline exit: %v", pline.WhyStopped())
}
}()
```

## Dependencies

None

## Urgency

Low - I haven't heard of other developers in the community complain of this issue.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.