aws / aws/sagemaker-python-sdk

Error message for Class PysparkProcessor--> get_run_args, when submit_py_files is not list is misleading

Đang mở
#3,809 0 bình luận 0 reaction 1 người được giao Được @nargokul nhận Xem trên GitHub
component: processing type: bug type: logging/error reporting
Ngôn ngữ chính
Python
Star
2.3k
Fork
1.3k
Merge trung bình
1 ngày 22 giờ
Pull request đã merge (30 ngày)
35

Mô tả

**Describe the bug**
When using a PysparkProcessor's class and methods, a deceptive error message is thrown. The class method PysparkProcessor--> get_run_args gives the following error when pipeline.upsert(role_arn= role) is called when one of the arguments, submit_py_files, is supplied and is not a Python list. The fault was initially thought to be a Sagemaker pipeline error, but we later learned that it was a PySpark processing step problem.

Error throw when called pipeline.upsert(role_arn= role). Consumed lot of our time

### "“PermissionError: [Errno 13] Permission denied: ‘/opt/.sagemakerinternal/conda/pkgs/brotlipy-0.7.0-py37h27cfd23_10039ghk6y6y’“.

**To reproduce**
1. Create a Pyspark processor
2. Pass in the the optional input submit_py_files as a string not a python list.
3. Create the Sagemaker pipeline with one step called preprocess.
4. Call pipeline.upsert(role_arn= role) and get this error.

**Expected behavior**
The PysparkProcessor class should have an isinstance() check for all the inputs to ensure the right datatypes. Even the doc string in the code is good.

**Screenshots or logs**
"“PermissionError: [Errno 13] Permission denied: ‘/opt/.sagemakerinternal/conda/pkgs/brotlipy-0.7.0-py37h27cfd23_10039ghk6y6y’“. When creating a Sagemaker pipeline, especially upserting the role, pipeline.upsert(role_arn= role)."

**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: 2.145.0
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: PysparkProcessor
- **Framework version**: 2.4
- **Python version**: 3.9
- **CPU or GPU**: CPU
- **Custom Docker image (Y/N)**:N

**Additional context**
Add any other context about the problem here.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.