aws / aws/amazon-sagemaker-feedback

Support LambdaStep for local execution

Open
#109 0 comments 0 reactions 0 assignees View on GitHub
feature-request
Dominant language
No language data
Stars
10
Forks
3
PR merge metrics
No merged PRs in 30d

Description

### Product Version

- [ ] Amazon SageMaker Studio Classic
- [ ] Amazon SageMaker Studio
- [X] It is not related to SageMaker Studio

### Product Category

Pipelines

### Description

I am using SageMaker [Local Mode](https://docs.aws.amazon.com/sagemaker/latest/dg/pipelines-local-mode.html) and it really speeds up my development cycle. I can run a dev pipeline in < 1 min instead of > 10 mins. However, the lacking support fot [Lambda Steps](https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-steps.html#step-type-lambda) in Local Mode breaks my efficient development workflow now that I want to add a lambda function. My use case is that I need a quick computation layer 'between two steps' (also see https://github.com/aws/sagemaker-python-sdk/issues/4676) and I don't want to use a processing step that has 2 mins startup time.

I hence request to add support for the Lambda Step in Local mode. Implementation wise, I'd just call the lambda function the same way it works in the non-local mode. I think calling a lambda function should be relatively simple compared to other steps that are already implemented. To further simplify a first iteration, you could only support existing lambda functions, i.e. don't support Lamda Functions that are created on the fly with the pipeline definition.

This is my first feedback here and I am very grateful for this repo. Looks like you already shipped some user requests, very promising. I [previously complained](https://github.com/aws/sagemaker-python-sdk/issues/4380) in the sagemaker-python-sdk repo about how community feedback is handled - was not aware of this repo here. I also have a few [other suggestions](https://github.com/aws/sagemaker-python-sdk/issues?q=is%3Aissue+author%3Alorenzwalthert+Local+Mode+) how Local Mode could be improved that I will post here at a later date (or maybe you can transfer them?)

### Other Details

_No response_

Contributor guide

Open the contributing guide

Research direction

The issue names SageMaker Local Mode pipelines and Lambda Steps but provides no repository files, tests, or entry point. Start by locating the local-mode step implementation and existing Lambda Step behavior in the SageMaker SDK, then verify that an existing Lambda function can run between local pipeline steps without requiring function creation in the pipeline.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.