aws / aws/sagemaker-huggingface-inference-toolkit
HF_TASK Enviournment Variable error
- Dominant language
- Python
- Stars
- 270
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
I have tried to deploy an embedding model in AWS Sagemaker endpoint using the provided guide that uses inference.py to deploy custom code. the endpoint is created and starts but when I query the end-point I get the following error.
"You need to define one of the following ['audio-classification', 'automatic-speech-recognition', 'feature-extraction', 'text-classification', 'token-classification', 'question-answering', 'table-question-answering', 'fill-mask', 'summarization', 'translation', 'text2text-generation', 'text-generation', 'zero-shot-classification', 'conversational', 'image-classification', 'image-segmentation', 'object-detection'] as env 'HF_TASK’
How do I set the enviournment variable in the note book I am using for deployment or inside the inference.py file.
Also is it necessary to keep ‘model_fn’ and ‘predict_fn’ name the same in the inference.py file
please guide I have been stuck for quite a while now and changing anything in the inference.py files requires me to generate the .tar.gz file again which takes alot of time
Contributor guide
Research direction
Start with the deployment guide, the notebook used to create the SageMaker endpoint, and inference.py, including the model_fn and predict_fn entry points. Reproduce the endpoint query error and trace how HF_TASK is supplied; done means the deployment path clearly identifies the required configuration and answers whether those function names are fixed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100