aws / aws/amazon-sagemaker-examples
[Example Request] - HPO with Spark and R algo
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Describe the use case example you want to see**
Hyperparameter optimization using spark and custom R algorithm
**Describe which SageMaker services are involved**
This would likely require a custom Docker container as well as spark.
Note, I see the tune_r_bring_your_own.ipynb as well as the pyspark_mnist_custom_estimator.ipynb scripts and would likely model an example off of these. However, I am unsure if PySpark would accept an image written in R.
Any insight into how one could approach this would be greatly appreciated!
Contributor guide
Research direction
Start by comparing tune_r_bring_your_own.ipynb and pyspark_mnist_custom_estimator.ipynb, the only entry points named in the issue. Clarify whether the requested Spark, custom R algorithm, and hyperparameter optimization combination is supported, then define an end-to-end example and its required container setup as the completion criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker, jupyter-notebook, r, spark
- Domain
- data-engineering, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100