aws / aws/sagemaker-python-sdk

Allow storing inputs and outputs of FrameworkProcessor cross-accounts

Đang mở
#2,913 1 bình luận 3 reaction 1 người được giao Được @nargokul nhận Xem trên GitHub
component: processing type: feature request
Ngôn ngữ chính
Python
Star
2.3k
Fork
1.3k
Merge trung bình
1 ngày 22 giờ
Pull request đã merge (30 ngày)
35

Mô tả

**Describe the feature you'd like**
Definitions:
- Account A: where I will run SageMaker FrameworkProcessor jobs.
- Account B: where data exists that I need to access for `ProcessingInput.source` and `ProcessingOutput.destination`
- Role A: IAM role in account A that has `SageMakerFullAccess` policy attached.
- Role B: IAM role in acount B that has S3 read and write access to the needed data.
Role A can assume role B, and role B has a trusted relationship with Role A.

Problem:
I would like to create a `FrameworkProcessor` (specifically a `TensorFlowProcessor`) instance that can run on Account A but read and write data to Account B to avoid having to copy data back and forth between the two accounts.

**How would this feature be used? Please describe.**
A `role` parameter could be added to the `ProcessingInput` and `ProcessingOutput` classes that would be assumed before accessing the data.
```
processor = TensorFlowProcessor(role=role_A,...)
processor.run(
inputs=[ProcessingInput(source=.., destination=.., role=role_B)],
outputs=[ProcessingOutput(source=..,destination=.., role=role_B)],
...
)
```

**Describe alternatives you've considered**
There is a `role` parameter in the `TensorFlowProcessor` constructor. However,

- Using Role A fails with error ` No S3 objects found under S3 URL...`: reason is object exists in Account B not A!
- Using Role B fails with error:

`botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreateProcessingJob operation: RoleArn: Cross-account pass role is not allowed.`

Reason: we need the SageMaker permissions on Account A to be defined in the role.

How can I tell SageMaker to use Role A for creating and running the processing job but to assume role B to access the datasets in account B? For example, I am able to do that easily in SageMaker notebooks.

Please let me know if there is a way to achieve that with the current `FrameworkProcessor` implementation.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.