aws / aws/sagemaker-huggingface-inference-toolkit
Can’t create an inference service for models that depend on packages i.e. `espeak`
- Dominant language
- Python
- Stars
- 270
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to deploy an inference endpoint for a speech-to-IPA model on SageMaker, using HuggingFace's toolkit for "facebook/wav2vec2-lv-60-espeak-cv-ft", which depends on `espeak` (`sudo yum install speak`) to perform some speech conversions. The VM that the service spins up doesn't contain the option for me to input a specific package to be installed or a script to do so.
```
ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
"code": 400,
"type": "InternalServerException",
"message": "espeak not installed on your system"
}```
Contributor guide
Research direction
Start at the SageMaker inference endpoint setup and the VM startup path described in the issue. Trace how the HuggingFace toolkit configures the service environment and whether it provides a package or script hook. Done means a model requiring espeak can be deployed with its dependency installed and the endpoint no longer reports “espeak not installed on your system.”
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, huggingface, python
- Domain
- cloud, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100