aws / aws/sagemaker-huggingface-inference-toolkit

HF_TASK Enviournment Variable error

Open
#110 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
270
Forks
60
PR merge metrics
No merged PRs in 30d

Description

I have tried to deploy an embedding model in AWS Sagemaker endpoint using the provided guide that uses inference.py to deploy custom code. the endpoint is created and starts but when I query the end-point I get the following error.
"You need to define one of the following ['audio-classification', 'automatic-speech-recognition', 'feature-extraction', 'text-classification', 'token-classification', 'question-answering', 'table-question-answering', 'fill-mask', 'summarization', 'translation', 'text2text-generation', 'text-generation', 'zero-shot-classification', 'conversational', 'image-classification', 'image-segmentation', 'object-detection'] as env 'HF_TASK’

How do I set the enviournment variable in the note book I am using for deployment or inside the inference.py file.
Also is it necessary to keep ‘model_fn’ and ‘predict_fn’ name the same in the inference.py file

please guide I have been stuck for quite a while now and changing anything in the inference.py files requires me to generate the .tar.gz file again which takes alot of time

Contributor guide

Open the contributing guide

Research direction

Start with the deployment guide, the notebook used to create the SageMaker endpoint, and inference.py, including the model_fn and predict_fn entry points. Reproduce the endpoint query error and trace how HF_TASK is supplied; done means the deployment path clearly identifies the required configuration and answers whether those function names are fixed.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.