Field name validation
- Langage dominant
- Scala
- Étoiles
- 33
- Forks
- 16
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
## Background
If a field name for standardization is not valid for _parquet_ format, the whole standardization is executed and the process fails on saving to _parquet_ file.
## Feature
It would be much more efficient to verify the column names as part of the verification at standardization start and fail early.
## Proposed Solution
Possible places to do the validation at:
* `SchemaValidator` object
* `FieldValidator` class
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Start by inspecting the `SchemaValidator` object and `FieldValidator` class to determine where standardization-start verification is performed. Reproduce the invalid parquet field-name case, then add validation before standardization proceeds and verify that invalid names fail early rather than only when saving the parquet file.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- scala, spark
- Domaine
- data-engineering
- Type d'issue
- Fonctionnalité
- Difficulté
- 3/5
- Temps estimé
- 1-2 jours
- Activité
- À l'abandon
- Clarté
- Plutôt claire
- Accessibilité débutants
- 42/100