aws / aws/amazon-sagemaker-examples

Deepar Notebook - Different outputs for different runs despite setting the seed.

Open
#1,058 7 comments 1 reaction 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Hi,

I've observed a problem of instability with the DeepAr algorithm as of deepar-electricity notebook. Basically when running twice, despite setting both a numpy seed and random seed, the error output changes, often even largely.

The issue can be reproduced by running the notebook twice and taking note of the cross-validation out using `estimator.training_job_analytics.dataframe()`

How is that possible?

Contributor guide

Open the contributing guide

Research direction

Run the deepar-electricity notebook twice with the stated NumPy and random seeds, then compare the cross-validation output from estimator.training_job_analytics.dataframe(). Trace the notebook’s DeepAR training setup; done when the differing output is explained and the example has a verified, repeatable result.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, python
Domain
data, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.