AbsaOSS / AbsaOSS/dataset-comparison

Add switch to fail on duplicate rows

Open
#4 0 comments 0 reactions 0 assignees View on GitHub
enhancement low
Dominant language
Scala
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Description

# Description

Some datasets are with duplicate rows, in some it is a major issues, mainly if they do the a (Composite) Primary Key defined. In those cases users would opt to fail fast if duplicates are found.

# Feature

Add a switch to input params to fail if input `A` or `B` has duplicate rows.

> Original author @Zejnilovic

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.