awslabs / awslabs/python-deequ

ColumnProfilerRunner of Pydeequ is not working appropriatley

Offen
#43 4 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @jaoanan1126 Auf GitHub ansehen
question stale
Vorherrschende Sprache
Jupyter Notebook
Sterne
826
Forks
158
Ø Merge
9 T. 22 Std.
Gemergte PRs (30 T.)
3

Beschreibung

**Describe the bug**
ColumnProfilerRunner of Pydeequ is not working on any of the data that I used for test. I've used it on both internal data and also the custom created Data Frame that was given in the Pydeequ documentation. I ran the job in AWS Glue ETL.

The underlying Deequ jar version that we are using is: **deequ-1.0.2.jar**

I believe that the Pydeequ version we are using is **0.1.5**.

**Expected behavior**
I would expect the ColumnProfilerRunner of Pydeequ should give the profiling results. Instead, it is failing with:
`py4j.protocol.Py4JError: An error occurred while calling o226.kll. Trace:
py4j.Py4JException: Method kll([]) does not exist`

**Screenshots**
Screenshot from the Glue ETL logs:
![image](https://user-images.githubusercontent.com/5527761/121947222-d1107c00-cd23-11eb-9de8-7cdb7e05427f.png)

**Additional context**
ConstraintSuggestionRunner and Analyzers are working fine on the same data that I have used and in the same environment that I ran the code in.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.