AbsaOSS / AbsaOSS/enceladus

Create runs for streaming jobs originating from Hyperdrive

未关闭
#1,214 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
Conformance feature priority: undecided under discussion
主要语言
Scala
星标
33
派生
16
PR 合并指标
30 天内没有已合并 PR

描述

## Background
Now that Conformance can be run on streaming jobs from Hyperdrive we need to decide how we want to track these jobs as runs.

Currently, no runs are created for a streaming conformance job.

## Feature
We can create a separate entity for streaming jobs. My proposal is to call it `transmission`. It would contain only parameters that are relevant for streaming, such as
* Status (such as `running`, `stopped`, etc).
* A start timestamp of a streaming job
* A finish timestamp of a streaming job
* Number of records processed
* Average throughput
* Topic names
* Hyperdrive version and configuration options

Transmissions should have a separate tab in Menas where users can see what's happening.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。