awslabs / awslabs/python-deequ

[BUG] Holt Winters Anomaly Strategy fails due to `NoClassDefFoundError`

Open
#226 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Jupyter Notebook
Stars
826
Forks
158
Avg merge
9d 22h
Merged PRs (30d)
3

Description

**Holt Winters Anomaly Strategy fails due to `NoClassDefFoundError`**

**To Reproduce**
Steps to reproduce the behavior:
1. Replicating [Glue tutorial](https://github.com/awslabs/python-deequ/blob/master/tutorials/glue_tutorials/advanced_anomaly_detection_tutorial.ipynb) for the anomaly detection using Holt Winters Anomaly Strategy.
2. Attempted with the following combinations of versions:
- AWS Glue version 4, Spark 3.3, pydeequ 1.4.0, deequ jar com.amazon.deequ:deequ:2.0.7-spark-3.3.
- AWS Glue version 3, Spark 3.1, pydeeqy 1.4.0, deequ jar com.amazon.deequ:deequ:2.0.7-spark-3.1
4. I ran the code below after generating the metrics for years 2013 and 2014:
```
for month in range(1,9):
date = "\'2015" +'/'+str(month)+"\'"
df = df_2015.filter("review_date =" + date)
key_tags = {'tag': date}
result_key_2015 = ResultKey(session, ResultKey.current_milli_time(), key_tags)

jewelry_result = VerificationSuite(session).onData(df)\
.useRepository(metricsRepository) \
.saveOrAppendResult(result_key_2015) \
.addAnomalyCheck(HoltWinters(session, MetricInterval.Monthly, SeriesSeasonality.Yearly), Sum('total_votes'))\
.run()
```

5. I get the following error:

```
Py4JJavaError: An error occurred while calling o560.run.
: java.lang.NoClassDefFoundError: breeze/stats/package$
at com.amazon.deequ.anomalydetection.seasonal.HoltWinters.detect(HoltWinters.scala:243)
at com.amazon.deequ.anomalydetection.AnomalyDetector.detectAnomaliesInHistory(AnomalyDetector.scala:98)
.....
```

**Expected behavior**
The expected behavior is to obtain a message that anomaly has been detected.

What can be done to fix this error? Thank you.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.