aws / aws/amazon-sagemaker-examples

[Example Request] - HPO with Spark and R algo

Open
#2,171 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

**Describe the use case example you want to see**
Hyperparameter optimization using spark and custom R algorithm

**Describe which SageMaker services are involved**
This would likely require a custom Docker container as well as spark.

Note, I see the tune_r_bring_your_own.ipynb as well as the pyspark_mnist_custom_estimator.ipynb scripts and would likely model an example off of these. However, I am unsure if PySpark would accept an image written in R.

Any insight into how one could approach this would be greatly appreciated!

Contributor guide

Open the contributing guide

Research direction

Start by comparing tune_r_bring_your_own.ipynb and pyspark_mnist_custom_estimator.ipynb, the only entry points named in the issue. Clarify whether the requested Spark, custom R algorithm, and hyperparameter optimization combination is supported, then define an end-to-end example and its required container setup as the completion criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, jupyter-notebook, r, spark
Domain
data-engineering, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.