aws-samples / aws-samples/sagemaker-101-workshop

Demonstrate script testing/debugging in the script mode walkthroughs

Open
#33 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
80
Forks
51
PR merge metrics
No merged PRs in 30d

Description

### Feature request

We already provide example shell commands to invoke and test training scripts for all variants of the [migration challenge](https://github.com/aws-samples/sagemaker-101-workshop/tree/main/migration_challenge). For example [from SKLearn](https://github.com/aws-samples/sagemaker-101-workshop/blob/58ad6f83eeabf978b72f5ca85c0a11b776fc116e/migration_challenge/sklearn_cls/Instructions.ipynb):

```
!python3 src/main.py \
--train ./data/train \
--test ./data/test \
--model_dir ./data/model \
--class_names {class_names_str} \
--n_estimators=100 \
--min_samples_leaf=3
```

**Since this is the recommended debugging workflow, we should also demonstrate it in the [script mode walkthroughs](https://github.com/aws-samples/sagemaker-101-workshop/tree/main/custom_script_demos)** by adding equivalent commands in the 'SageMaker' variants of these notebooks - before the `Estimator` gets created.

This will help these notebooks illustrate the process/workflow of translating from in-notebook to notebook+job, better than just showing the final result.

### Background

Today, we use in-notebook shell commands as the recommended script debugging workflow for the migration challenge - because our options are somewhat constrained for a workshop:

- SageMaker Warm Pools [requires a quota increase](https://docs.aws.amazon.com/sagemaker/latest/dg/train-warm-pools.html#train-warm-pools-resource-limits) to enable
- SageMaker Local Mode isn't [natively supported in Studio](https://repost.aws/knowledge-center/sagemaker-studio-local-mode) at this time
- [SageMaker SSH Helper](https://github.com/aws-samples/sagemaker-ssh-helper) has a bit of a learning curve for any data scientists not already familiar with online SSH-based debugging (and some setup required - although we could probably automate that).

We talk about these other options in the post-challenge wrap-up, but don't want to confuse the issue by introducing them up-front in the code.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.