aws / aws/sagemaker-python-sdk

Allow storing inputs and outputs of FrameworkProcessor cross-accounts

Abierto
#2,913 1 comentario 3 reacciones 1 asignado Reclamado por @nargokul Ver en GitHub
component: processing type: feature request
Lenguaje dominante
Python
Estrellas
2.3k
Forks
1.3k
Merge medio
1 d 22 h
PR fusionados (30 d)
35

Descripción

**Describe the feature you'd like**
Definitions:
- Account A: where I will run SageMaker FrameworkProcessor jobs.
- Account B: where data exists that I need to access for `ProcessingInput.source` and `ProcessingOutput.destination`
- Role A: IAM role in account A that has `SageMakerFullAccess` policy attached.
- Role B: IAM role in acount B that has S3 read and write access to the needed data.
Role A can assume role B, and role B has a trusted relationship with Role A.

Problem:
I would like to create a `FrameworkProcessor` (specifically a `TensorFlowProcessor`) instance that can run on Account A but read and write data to Account B to avoid having to copy data back and forth between the two accounts.

**How would this feature be used? Please describe.**
A `role` parameter could be added to the `ProcessingInput` and `ProcessingOutput` classes that would be assumed before accessing the data.
```
processor = TensorFlowProcessor(role=role_A,...)
processor.run(
inputs=[ProcessingInput(source=.., destination=.., role=role_B)],
outputs=[ProcessingOutput(source=..,destination=.., role=role_B)],
...
)
```

**Describe alternatives you've considered**
There is a `role` parameter in the `TensorFlowProcessor` constructor. However,

- Using Role A fails with error ` No S3 objects found under S3 URL...`: reason is object exists in Account B not A!
- Using Role B fails with error:

`botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreateProcessingJob operation: RoleArn: Cross-account pass role is not allowed.`

Reason: we need the SageMaker permissions on Account A to be defined in the role.

How can I tell SageMaker to use Role A for creating and running the processing job but to assume role B to access the datasets in account B? For example, I am able to do that easily in SageMaker notebooks.

Please let me know if there is a way to achieve that with the current `FrameworkProcessor` implementation.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.