aws / aws/sagemaker-huggingface-inference-toolkit

SageMaker deployment errors

Open
#94 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
270
Forks
60
PR merge metrics
No merged PRs in 30d

Description

# Background

We are attempting to deploy SageMaker Endpoints using the code provided under Deploy - Amazon SageMaker from huggingface.co for these two models:

https://huggingface.co/Salesforce/codegen25-7b-multi
https://huggingface.co/openchat/opencoderplus

# Error

Both endpoints consistently fail to deploy. Both fail health checks - error logs available on request as it does not appear I can attach them here.

Contributor guide

Open the contributing guide

Research direction

Start with the Deploy - Amazon SageMaker example on huggingface.co and reproduce the endpoint deployments for the two named models. Obtain and inspect the missing health-check error logs, then compare the deployment behavior for both models. Done means identifying the deployment failure and confirming that the affected endpoints pass health checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, huggingface, python
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.