aws / aws/amazon-sagemaker-feedback
Support LambdaStep for local execution
- Dominant language
- No language data
- Stars
- 10
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
### Product Version
- [ ] Amazon SageMaker Studio Classic
- [ ] Amazon SageMaker Studio
- [X] It is not related to SageMaker Studio
### Product Category
Pipelines
### Description
I am using SageMaker [Local Mode](https://docs.aws.amazon.com/sagemaker/latest/dg/pipelines-local-mode.html) and it really speeds up my development cycle. I can run a dev pipeline in < 1 min instead of > 10 mins. However, the lacking support fot [Lambda Steps](https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-steps.html#step-type-lambda) in Local Mode breaks my efficient development workflow now that I want to add a lambda function. My use case is that I need a quick computation layer 'between two steps' (also see https://github.com/aws/sagemaker-python-sdk/issues/4676) and I don't want to use a processing step that has 2 mins startup time.
I hence request to add support for the Lambda Step in Local mode. Implementation wise, I'd just call the lambda function the same way it works in the non-local mode. I think calling a lambda function should be relatively simple compared to other steps that are already implemented. To further simplify a first iteration, you could only support existing lambda functions, i.e. don't support Lamda Functions that are created on the fly with the pipeline definition.
This is my first feedback here and I am very grateful for this repo. Looks like you already shipped some user requests, very promising. I [previously complained](https://github.com/aws/sagemaker-python-sdk/issues/4380) in the sagemaker-python-sdk repo about how community feedback is handled - was not aware of this repo here. I also have a few [other suggestions](https://github.com/aws/sagemaker-python-sdk/issues?q=is%3Aissue+author%3Alorenzwalthert+Local+Mode+) how Local Mode could be improved that I will post here at a later date (or maybe you can transfer them?)
### Other Details
_No response_
Contributor guide
Research direction
The issue names SageMaker Local Mode pipelines and Lambda Steps but provides no repository files, tests, or entry point. Start by locating the local-mode step implementation and existing Lambda Step behavior in the SageMaker SDK, then verify that an existing Lambda function can run between local pipeline steps without requiring function creation in the pipeline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100