aws / aws/amazon-sagemaker-examples
Deepar Notebook - Different outputs for different runs despite setting the seed.
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
Hi,
I've observed a problem of instability with the DeepAr algorithm as of deepar-electricity notebook. Basically when running twice, despite setting both a numpy seed and random seed, the error output changes, often even largely.
The issue can be reproduced by running the notebook twice and taking note of the cross-validation out using `estimator.training_job_analytics.dataframe()`
How is that possible?
Contributor guide
Research direction
Run the deepar-electricity notebook twice with the stated NumPy and random seeds, then compare the cross-validation output from estimator.training_job_analytics.dataframe(). Trace the notebook’s DeepAR training setup; done when the differing output is explained and the example has a verified, repeatable result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook, python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100