AbsaOSS / AbsaOSS/enceladus

Schema validation exception does not include detailed error message

Abierto
#1,046 0 comentarios 0 reacciones 0 asignados Ver en GitHub
bug priority: undecided Standardization
Lenguaje dominante
Scala
Estrellas
33
Forks
16
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Describe the bug
If an inconsistency is found in a schema definition, Standardization logs a validation error, and then throws an exception that does not contain details of the validation failure.
This makes it hard for a user to investigate schema validation issues.

```
19/11/19 10:49:53 ERROR SchemaChecker$: Validation error for column 'long_type', pattern '': 'FF' cannot be cast to long
19/11/19 10:49:53 INFO RestClient: POST - URL: http://localhost:8080/menas/api/runs/updateRunStatus/548b92d8-4028-42cd-a77e-4d62e9babf29
19/11/19 10:49:53 INFO RestClient: Request Body: ***
19/11/19 10:49:53 INFO StandardizationJob$: Menas API Run URL: http://localhost:8080/menas/api/runs/NumPatternsTest/1/5
19/11/19 10:49:53 INFO StandardizationJob$: Menas UI Run URL: http://localhost:8080/menas/#/runs/NumPatternsTest/1/5
19/11/19 10:49:53 INFO StandardizationJob$: Menas API Run URL: http://localhost:8080/menas/api/runs/NumPatternsTest/1/5
19/11/19 10:49:53 INFO StandardizationJob$: Menas UI Run URL: http:/localhost:8080/menas/#/runs/NumPatternsTest/1/5
Exception in thread "main" za.co.absa.enceladus.utils.validation.ValidationException: A fatal schema validation error occurred.
at za.co.absa.enceladus.standardization.interpreter.StandardizationInterpreter$.validateSchemaAgainstSelfInconsistencies(StandardizationInterpreter.scala:69)
at za.co.absa.enceladus.standardization.interpreter.StandardizationInterpreter$.standardize(StandardizationInterpreter.scala:45)

```
## To Reproduce
Run a Standardization job with a schema that contains invalid metadata. Example:
```json
{
"type": "struct",
"fields": [
{
"name": "long_type",
"type": "long",
"nullable": false,
"metadata": {
"no_radix": "16",
"default": "FF"
}
},
{
"name": "double_type",
"type": "double",
"nullable": true,
"metadata": {
"pattern": "0.#"
}
},
{
"name": "decimal_type",
"type": "decimal(18,2)",
"nullable": false,
"metadata": {
"pattern": "0.#",
"decimal_separator": ",",
"grouping_separator": "."
}
}
]
}
```

## Expected behaviour
The validation exception should contain more details.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.