AbsaOSS / AbsaOSS/enceladus

When updating schema on dataset there is no check for collisions with conformance rules

Ouverte
#715 2 commentaires 1 réaction 0 personnes assignées Voir sur GitHub
bug Menas priority: medium
Langage dominant
Scala
Étoiles
33
Forks
16
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

## Describe the bug
If I have a schema with columns `a`,`b`,`c`, create a column `d` (string) using conformance rule and then update schema to contain `d` (timestamp) column of different kind and change it for the dataset, there is nothing stopping me and the mistake will be only evident in the middle of conformance when it tells me there already is column `d`.

## To Reproduce
Steps to reproduce the behavior OR commands run:
1. Have dataset `X` with columns `a`,`b`,`c`
2. Add conformance rule that creates columns `d`
3. Change schema of dataset `X` to have column `d` from raw.
4. Run standardization and conformance
5. See error from conformance in console/logger/STDOUT

## Expected behaviour
When changing/updating schema check if it is possible depending on output columns of conformance rules.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.