AbsaOSS / AbsaOSS/enceladus

Add an ability to run Conformance on a streaming source

Open
#1,014 2 comments 0 reactions 1 assignee Claimed by @yruslan View on GitHub
Conformance feature priority: medium under discussion
Dominant language
Scala
Stars
33
Forks
16
PR merge metrics
No merged PRs in 30d

Description

## Background
Add streaming support for Dynamic Conformance requires changes in the way we handle inputs, outputs and control measurements.

## Feature
Add initial support for running Conformance for a streaming source.
* Add command line parameters to identify that the job is streaming, specify the checkpoint location, etc.
* Change the way we load data to create dataframes.
* Change the way we save the output data.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.