aws / aws/sagemaker-python-sdk

Sagemaker Local-Mode permission errors

Đang mở
#4,764 2 bình luận 0 reaction 1 người được giao Được @pintaoz-aws nhận Xem trên GitHub
type: bug
Ngôn ngữ chính
Python
Star
2.3k
Fork
1.3k
Merge trung bình
1 ngày 22 giờ
Pull request đã merge (30 ngày)
35

Mô tả

**Describe the bug**
I keep running into issues trying to run bring your own container (BYOC) images, they either work in local model, or they work in "cloud mode" but not both. The issue is almost certainly docker permission problems but I cannot see a way of resolving.

**To reproduce**
I have a docker image

```
FROM continuumio/miniconda3

WORKDIR /opt/ml/code/
COPY src/ /opt/ml/code/

ENTRYPOINT ["bash", "/opt/ml/code/processor.sh"]
```

The `processor.sh` creates a conda venv, installs packages into it then runs `conda pack` with the output set to `/opt/ml/processing/output/pyenv.tar.gz`

In `local` mode this step fails on this line

https://github.com/aws/sagemaker-python-sdk/blob/0e3769ea70a9b4c471a2bfb3a0f6116b9958361e/src/sagemaker/local/utils.py#L84

Because the container user (root) writes the file with permissions that are too restrictive. Note there's a "fix" on line 99 for a similar issue but it doesn't help.

I tried to work around this by adding a non-root (gid/pid 1000) user to the container. This "fixed" the local issue - but now the contain fails when I run it normally. It appears to be the opposite issue, the container user doesn't have access to the volume ` /opt/ml/processing/output/` so the script fails.

**Expected behavior**
Need to be able to run pipelines in local and sagemaker mode without permission errors. It seems that in local mode, scripts need to run with the same pid as the host user, but in sagemaker mode they need to run as root? Is there some way they can work in both?

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.