Field name validation
- Lenguaje dominante
- Scala
- Estrellas
- 33
- Forks
- 16
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
## Background
If a field name for standardization is not valid for _parquet_ format, the whole standardization is executed and the process fails on saving to _parquet_ file.
## Feature
It would be much more efficient to verify the column names as part of the verification at standardization start and fail early.
## Proposed Solution
Possible places to do the validation at:
* `SchemaValidator` object
* `FieldValidator` class
Guía de contribución
Línea de trabajo
Start by inspecting the `SchemaValidator` object and `FieldValidator` class to determine where standardization-start verification is performed. Reproduce the invalid parquet field-name case, then add validation before standardization proceeds and verify that invalid names fail early rather than only when saving the parquet file.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- scala, spark
- Área
- data-engineering
- Tipo de issue
- Nueva funcionalidad
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 42/100