aws / aws/amazon-sagemaker-examples

[Content Improvement] Improve reproducability in sagemaker studio

Open
#4,391 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

**Link to the notebook**
[Add the link to the notebook.](https://github.com/aws/amazon-sagemaker-examples/blob/main/inference/nlp/realtime/triton/multi-model/t5_pytorch_python-backend/t5_pytorch_python-backend.ipynb)

**What aspects of the notebook can be improved?**
This and other notebooks state ` Note This notebook was tested with the `conda_python3` kernel on an Amazon SageMaker notebook instance of type `g5.xlarge`.`

Its not at all clear what`conda_python3` refers to? In my Sagemaker Studio there are two drops downs "Image" and "Kernel" - the only option for "Kernel" is "Python 3" and as far as I can see "Data Science 3" seems to be based on conda but it doesn't work with this and many other examples so either it's not `conda_python3` or something is out of date.

In particular the first cell

```
!pip install -qU pip awscli boto3 sagemaker
!pip install nvidia-pyindex --quiet
!pip install tritonclient[http] --quiet
!pip install transformers[sentencepiece] --quiet
```

results in

```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
distributed 2022.7.0 requires tornado<6.2,>=6.0.3, but you have tornado 6.3.2 which is incompatible.
```

And then the next cell also fails

```
import boto3, json, sagemaker, time
from sagemaker import get_execution_role
import numpy as np
import os

os.environ["TOKENIZERS_PARALLELISM"] = "false"

# sagemaker variables
role = get_execution_role()
sm_client = boto3.client(service_name="sagemaker")
runtime_sm_client = boto3.client("sagemaker-runtime")
sagemaker_session = sagemaker.Session(boto_session=boto3.Session())
s3_client = boto3.client("s3")
bucket = sagemaker.Session().default_bucket()
prefix = "nlp-mme-gpu"

# account mapping for SageMaker MME Triton Image
account_id_map = {
"us-east-1": "785573368785",
"us-east-2": "007439368137",
"us-west-1": "710691900526",
"us-west-2": "301217895009",
"eu-west-1": "802834080501",
"eu-west-2": "205493899709",
"eu-west-3": "254080097072",
"eu-north-1": "601324751636",
"eu-south-1": "966458181534",
"eu-central-1": "746233611703",
"ap-east-1": "110948597952",
"ap-south-1": "763008648453",
"ap-northeast-1": "941853720454",
"ap-northeast-2": "151534178276",
"ap-southeast-1": "324986816169",
"ap-southeast-2": "355873309152",
"cn-northwest-1": "474822919863",
"cn-north-1": "472730292857",
"sa-east-1": "756306329178",
"ca-central-1": "464438896020",
"me-south-1": "836785723513",
"af-south-1": "774647643957",
}

region = boto3.Session().region_name
if region not in account_id_map.keys():
raise ("UNSUPPORTED REGION")

base = "amazonaws.com.cn" if region.startswith("cn-") else "amazonaws.com"
mme_triton_image_uri = (
"{account_id}.dkr.ecr.{region}.{base}/sagemaker-tritonserver:23.02-py3".format(
account_id=account_id_map[region], region=region, base=base
)
)
```

```
/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:155: UserWarning: A NumPy version >=1.18.5 and <1.25.0 is required for this version of SciPy (detected version 1.25.1
warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
```

**What are your suggestions?**

Ensure that the documentation on which kernel is required matches what's available.

Ensure that dependencies are pinned for reliable reproduction.

Contributor guide

Open the contributing guide

Research direction

Start with inference/nlp/realtime/triton/multi-model/t5_pytorch_python-backend/t5_pytorch_python-backend.ipynb and run its installation and import cells in SageMaker Studio. Compare the documented conda_python3 kernel with the available Image and Kernel options, then update the notebook guidance and dependency installation for reproducible execution, including the reported compatibility warnings.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, python
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.