awslabs / awslabs/python-deequ
[BUG] Holt Winters Anomaly Strategy fails due to `NoClassDefFoundError`
- Dominant language
- Jupyter Notebook
- Stars
- 826
- Forks
- 158
- Avg merge
- 9d 22h
- Merged PRs (30d)
- 3
Description
**Holt Winters Anomaly Strategy fails due to `NoClassDefFoundError`**
**To Reproduce**
Steps to reproduce the behavior:
1. Replicating [Glue tutorial](https://github.com/awslabs/python-deequ/blob/master/tutorials/glue_tutorials/advanced_anomaly_detection_tutorial.ipynb) for the anomaly detection using Holt Winters Anomaly Strategy.
2. Attempted with the following combinations of versions:
- AWS Glue version 4, Spark 3.3, pydeequ 1.4.0, deequ jar com.amazon.deequ:deequ:2.0.7-spark-3.3.
- AWS Glue version 3, Spark 3.1, pydeeqy 1.4.0, deequ jar com.amazon.deequ:deequ:2.0.7-spark-3.1
4. I ran the code below after generating the metrics for years 2013 and 2014:
```
for month in range(1,9):
date = "\'2015" +'/'+str(month)+"\'"
df = df_2015.filter("review_date =" + date)
key_tags = {'tag': date}
result_key_2015 = ResultKey(session, ResultKey.current_milli_time(), key_tags)
jewelry_result = VerificationSuite(session).onData(df)\
.useRepository(metricsRepository) \
.saveOrAppendResult(result_key_2015) \
.addAnomalyCheck(HoltWinters(session, MetricInterval.Monthly, SeriesSeasonality.Yearly), Sum('total_votes'))\
.run()
```
5. I get the following error:
```
Py4JJavaError: An error occurred while calling o560.run.
: java.lang.NoClassDefFoundError: breeze/stats/package$
at com.amazon.deequ.anomalydetection.seasonal.HoltWinters.detect(HoltWinters.scala:243)
at com.amazon.deequ.anomalydetection.AnomalyDetector.detectAnomaliesInHistory(AnomalyDetector.scala:98)
.....
```
**Expected behavior**
The expected behavior is to obtain a message that anomaly has been detected.
What can be done to fix this error? Thank you.
Contributor guide
Assessment
This issue has not been assessed yet.