aws / aws/amazon-sagemaker-feedback

Support ManifestFile in SageMaker Local Mode

Open
#201 0 comments 0 reactions 0 assignees View on GitHub
feature-request
Dominant language
No language data
Stars
10
Forks
3
PR merge metrics
No merged PRs in 30d

Description

### Product Version

- [ ] Amazon SageMaker Studio Classic
- [ ] Amazon SageMaker Studio
- [x] It is not related to SageMaker Studio

### Product Category

Pipelines

### Description

According to the [docs](https://sagemaker.readthedocs.io/en/stable/overview.html#local-pipelines), it's not yet supported to run SageMaker Jobs (and pipelines) in local mode with ManifestFile. This breaks the efficient developer workflow, since I either
* use manifest files and not use local mode anymore.
* use standard file mode but don't get the benefits of the manifest file (i.e. can only specify a prefix from which to download files instead of specifying specific files).

It would be great to see official support for this. It seems the manifest file is currently just dropped in the container as if it was a `S3DataType: ' S3Prefix'`. So one can work around the limitation and download the linked files with the right logic.

### Other Details

I opened a support case for this: 175285324300144.

Contributor guide

Open the contributing guide

Research direction

Start with the local pipelines documentation linked in the issue, then trace how SageMaker Jobs and pipelines handle ManifestFile versus S3DataType 'S3Prefix' in local mode. Done means local-mode jobs and pipelines officially support ManifestFile and resolve the linked files instead of treating the manifest as a prefix.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.