aws / aws/sagemaker-spark

[Feature Request] Distributed inference with local mode

Open
#117 2 comments 0 reactions 0 assignees View on GitHub
feature request wontfix
Dominant language
Scala
Stars
301
Forks
129
PR merge metrics
No merged PRs in 30d

Description

Please fill out the form below.

### System Information
- **Spark or PySpark**: Spark
- **SDK Version**: N/A
- **Spark Version**: N/A
- **Algorithm (e.g. KMeans)**: N/A

### Describe the problem

Currently Python SageMaker SDK supports local mode.

```
import numpy
from sagemaker.mxnet import MXNetModel

model_location = 's3://mybucket/my_model.tar.gz'
code_location = 's3://mybucket/sourcedir.tar.gz'
image_url = get_image_uri(sess.boto_region_name, 'image', repo_version="latest")
s3_model = MXNetModel(model_data=model_location, role='SageMakerRole', image=image_url,
entry_point='mnist.py', source_dir=code_location)

predictor = s3_model.deploy(initial_instance_count=1, instance_type='local')
data = numpy.zeros(shape=(1, 1, 28, 28))
predictor.predict(data)

# Tear down the endpoint container and delete the corresponding endpoint configuration
predictor.delete_endpoint()

# Deletes the model
predictor.delete_model()
```

[And right now this SDK forces us to create endpoint when we define model.](https://github.com/aws/sagemaker-spark/blob/0f8b920a301c29c8adbdb451563fd8cf33177f8e/sagemaker-spark-sdk/src/main/scala/com/amazonaws/services/sagemaker/sparksdk/SageMakerModel.scala#L94-L96)

Is there any plan to support local mode for inferencing?

Contributor guide

Open the contributing guide

Research direction

Start by reading sagemaker-spark-sdk/src/main/scala/com/amazonaws/services/sagemaker/sparksdk/SageMakerModel.scala around lines 94-96, then compare its endpoint behavior with the Python SageMaker local-mode example in the issue. Clarify the expected Spark inference flow and define done as supporting local inference without forcing endpoint creation, including suitable validation coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, scala, spark
Domain
distributed-systems, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.