awslabs / awslabs/python-deequ

ColumnProfilerRunner of Pydeequ is not working appropriatley

Abierto
#43 4 comentarios 0 reacciones 1 asignado Ver en GitHub

@jaoanan1126 ya está trabajando en esto.

Desde el 8/7/2021.

question stale
Lenguaje dominante
Jupyter Notebook
Estrellas
826
Forks
158
Merge medio
9 d 22 h
PR fusionados (30 d)
3

Descripción

Describe the bug
ColumnProfilerRunner of Pydeequ is not working on any of the data that I used for test. I've used it on both internal data and also the custom created Data Frame that was given in the Pydeequ documentation. I ran the job in AWS Glue ETL.

The underlying Deequ jar version that we are using is: deequ-1.0.2.jar

I believe that the Pydeequ version we are using is 0.1.5.

Expected behavior
I would expect the ColumnProfilerRunner of Pydeequ should give the profiling results. Instead, it is failing with:
py4j.protocol.Py4JError: An error occurred while calling o226.kll. Trace: py4j.Py4JException: Method kll([]) does not exist

Screenshots
Screenshot from the Glue ETL logs:
image

Additional context
ConstraintSuggestionRunner and Analyzers are working fine on the same data that I have used and in the same environment that I ran the code in.

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.