aws / aws/sagemaker-python-sdk

`ProcessingS3Output`'s `s3_uri` to be an optional field

Open
#5,559 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.3k
Forks
1.3k
Avg merge
1d 22h
Merged PRs (30d)
35

Description

**Describe the feature you'd like**
In the SDK v2, `ProcessingOutput` has a `destination` parameter to be either an S3 path where users want to write their results, but they can set it to `None` when they want to delegate the storage to SageMaker. This option stores the results in an S3 prefix partitioned by `job_name`, `step_name`, and `output_name`, which is quite convenient for traceability and to store intermediate results of a pipeline.

**How would this feature be used? Please describe.**
Much like v2, v3 should also support this feature. Users may not want to fix the destination path for intermediate results, so ideally the same behaviour should apply to v3 by leaving `ProcessingS3Output`'s `s3_uri` field as `None`. Currently this field is mandatory and it's unclear whether users can reproduce the v2 `None` behaviour.

**Describe alternatives you've considered**
No alternatives considered so far other than sticking to v2.

**Additional context**
N/A

Contributor guide

Open the contributing guide

Research direction

Start by comparing the SDK v2 ProcessingOutput behavior with the v3 ProcessingS3Output definition and its handling of destination values. Confirm how an omitted S3 URI should be represented for intermediate outputs; done means v3 supports the documented v2-equivalent use case and verifies the behavior with appropriate tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.