awslabs / awslabs/python-deequ
ColumnProfilerRunner of Pydeequ is not working appropriatley
- Dominant language
- Jupyter Notebook
- Stars
- 826
- Forks
- 158
- Avg merge
- 9d 22h
- Merged PRs (30d)
- 3
Description
**Describe the bug**
ColumnProfilerRunner of Pydeequ is not working on any of the data that I used for test. I've used it on both internal data and also the custom created Data Frame that was given in the Pydeequ documentation. I ran the job in AWS Glue ETL.
The underlying Deequ jar version that we are using is: **deequ-1.0.2.jar**
I believe that the Pydeequ version we are using is **0.1.5**.
**Expected behavior**
I would expect the ColumnProfilerRunner of Pydeequ should give the profiling results. Instead, it is failing with:
`py4j.protocol.Py4JError: An error occurred while calling o226.kll. Trace:
py4j.Py4JException: Method kll([]) does not exist`
**Screenshots**
Screenshot from the Glue ETL logs:

**Additional context**
ConstraintSuggestionRunner and Analyzers are working fine on the same data that I have used and in the same environment that I ran the code in.
Contributor guide
Assessment
This issue has not been assessed yet.