aws / aws/sagemaker-python-sdk

`ProcessingS3Output`'s `s3_uri` to be an optional field

Open
#5,559 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.3k
Forks
1.3k
Avg merge
1d 22h
Merged PRs (30d)
35

Description

**Describe the feature you'd like**
In the SDK v2, `ProcessingOutput` has a `destination` parameter to be either an S3 path where users want to write their results, but they can set it to `None` when they want to delegate the storage to SageMaker. This option stores the results in an S3 prefix partitioned by `job_name`, `step_name`, and `output_name`, which is quite convenient for traceability and to store intermediate results of a pipeline.

**How would this feature be used? Please describe.**
Much like v2, v3 should also support this feature. Users may not want to fix the destination path for intermediate results, so ideally the same behaviour should apply to v3 by leaving `ProcessingS3Output`'s `s3_uri` field as `None`. Currently this field is mandatory and it's unclear whether users can reproduce the v2 `None` behaviour.

**Describe alternatives you've considered**
No alternatives considered so far other than sticking to v2.

**Additional context**
N/A

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.