AbsaOSS / AbsaOSS/enceladus

Track runs and their metrics for streaming jobs (Atum)

Open
#1,166 0 comments 0 reactions 0 assignees View on GitHub
Conformance feature priority: undecided under discussion
Dominant language
Scala
Stars
33
Forks
16
PR merge metrics
No merged PRs in 30d

Description

## Background
Currently, run objects are created for each batch job. Atum control measurements are made only for batch jobs as well.

## Feature
Need to decide how to track runs and control metrics for:
- Stream to batch ingestion + conformance (Hyperdrive)
- Stream to stream ingestion + conformance (Hyperdrive as well?)

```diff
- This task should be amended with a detailed design.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.