aws / aws/sagemaker-huggingface-inference-toolkit

Can’t create an inference service for models that depend on packages i.e. `espeak`

Open
#128 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
270
Forks
60
PR merge metrics
No merged PRs in 30d

Description

I'm trying to deploy an inference endpoint for a speech-to-IPA model on SageMaker, using HuggingFace's toolkit for "facebook/wav2vec2-lv-60-espeak-cv-ft", which depends on `espeak` (`sudo yum install speak`) to perform some speech conversions. The VM that the service spins up doesn't contain the option for me to input a specific package to be installed or a script to do so.

```
ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
"code": 400,
"type": "InternalServerException",
"message": "espeak not installed on your system"
}```

Contributor guide

Open the contributing guide

Research direction

Start at the SageMaker inference endpoint setup and the VM startup path described in the issue. Trace how the HuggingFace toolkit configures the service environment and whether it provides a package or script hook. Done means a model requiring espeak can be deployed with its dependency installed and the endpoint no longer reports “espeak not installed on your system.”

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, huggingface, python
Domain
cloud, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.