Example code to use sagemaker-spark for our own model
- Dominant language
- Scala
- Stars
- 301
- Forks
- 129
- PR merge metrics
- No merged PRs in 30d
Description
Please fill out the form below.
### System Information
- **Spark or PySpark**: PySpark
- **SDK Version**: Any or most recent
- **Spark Version**: Any or most recent
- **Algorithm (e.g. KMeans)**: Any - in our case, PyTorch models
### Describe the problem
Is there example code of sagemaker-spark to perform inference on a spark dataframe using our own ML model using a GPU inference endpoint spun up by sagemaker? In our case we are bringing over our own PyTorch model.
### Minimal repo / logs
Please provide any logs and a bare minimum reproducible test case, as this will be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
N/A
- **Exact command to reproduce**:
N/A
Contributor guide
Research direction
No files, tests, or entry points are named. Start by reviewing the repository's existing SageMaker Spark examples and inference entry points, then determine how a PySpark DataFrame can use a custom PyTorch model through a SageMaker GPU endpoint; done means a working example covering that flow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, pytorch
- Domain
- data-engineering, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100