awslabs / awslabs/python-deequ

Anomaly Check functions not defined when using PyDeequ on Glue

Aperta
#138 3 commenti 0 reazioni 1 assegnatario Rivendicata da @chenliu0831 Vedi su GitHub
bug question
Lingua principale
Jupyter Notebook
Stelle
826
Fork
158
Merge medio
9g 22h
PR unite (30g)
3

Descrizione

**Describe the bug**
While following this Tutorial:
https://github.com/awslabs/python-deequ/blob/master/tutorials/anomaly_detection.ipynb
Error:
```
File "", line 4, in
NameError: name 'RateOfChangeStrategy' is not defined
```
Same error for SimpleThresholdStrategy,RelativeRateOfChangeStrategy etc.

**To Reproduce**
Steps to reproduce the behavior:
1. Create a Glue Jobs with same code as per the Tutorial : https://github.com/awslabs/python-deequ/blob/master/tutorials/anomaly_detection.ipynb
2. Except, before importing pydeequ create python environment variable :
```
import os
os.environ["SPARK_VERSION"] = "3.3"
```

3. Use Glue Version 4.0 . Spark 3.3
4. To include Pydeequ module create a setuup.py as shown below :

```
from setuptools import setup

setup(
name="pydeequ_module",
version="0.1",
packages=['pydeequ_module'],
install_requires=['pydeequ==1.1.0rc0','sagemaker_pyspark']
)
```
Copy the .whl file to s3 location and refer it as additional python libraries in Glue job. Reference :
https://repost.aws/knowledge-center/glue-import-error-no-module-named
5. Use the dependent jar file as deequ-2.0.3-spark-3.3.jar
6. Run the job the error shows up as :NameError: name 'RelativeRateOfChangeStrategy' is not defined

**Expected behavior**
The job should succeed identifying anomaly.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.