[Feature Request] Distributed inference with local mode
- Dominant language
- Scala
- Stars
- 301
- Forks
- 129
- PR merge metrics
- No merged PRs in 30d
Description
Please fill out the form below.
### System Information
- **Spark or PySpark**: Spark
- **SDK Version**: N/A
- **Spark Version**: N/A
- **Algorithm (e.g. KMeans)**: N/A
### Describe the problem
Currently Python SageMaker SDK supports local mode.
```
import numpy
from sagemaker.mxnet import MXNetModel
model_location = 's3://mybucket/my_model.tar.gz'
code_location = 's3://mybucket/sourcedir.tar.gz'
image_url = get_image_uri(sess.boto_region_name, 'image', repo_version="latest")
s3_model = MXNetModel(model_data=model_location, role='SageMakerRole', image=image_url,
entry_point='mnist.py', source_dir=code_location)
predictor = s3_model.deploy(initial_instance_count=1, instance_type='local')
data = numpy.zeros(shape=(1, 1, 28, 28))
predictor.predict(data)
# Tear down the endpoint container and delete the corresponding endpoint configuration
predictor.delete_endpoint()
# Deletes the model
predictor.delete_model()
```
[And right now this SDK forces us to create endpoint when we define model.](https://github.com/aws/sagemaker-spark/blob/0f8b920a301c29c8adbdb451563fd8cf33177f8e/sagemaker-spark-sdk/src/main/scala/com/amazonaws/services/sagemaker/sparksdk/SageMakerModel.scala#L94-L96)
Is there any plan to support local mode for inferencing?
Contributor guide
Research direction
Start by reading sagemaker-spark-sdk/src/main/scala/com/amazonaws/services/sagemaker/sparksdk/SageMakerModel.scala around lines 94-96, then compare its endpoint behavior with the Python SageMaker local-mode example in the issue. Clarify the expected Spark inference flow and define done as supporting local inference without forcing endpoint creation, including suitable validation coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, scala, spark
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100