aws / aws/sagemaker-huggingface-inference-toolkit
Better Documentation for Custom Inference and HF_MODEL_ID
- Dominant language
- Python
- Stars
- 270
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
It's not at all clear if the `HuggingFaceModel` class from this library honors the `entry_point ` and `source_dir` when image_uri is specified. I have tried this with several containers and the only way I can run my custom `code/inference.py` is by archiving the model and providing the s3 path in ` model_data=s3_model_uri`. Providing the `HF_MODEL_ID` and `inference.py` should work but from the logs I can see that's not the case because there is nothing about loading the inference.py or when I make a call to the model my logs are not there but if I archive this model and upload to s3 it works.
This is not documented. It's also not intuitive.
Contributor guide
Research direction
Start with the HuggingFaceModel behavior around entry_point, source_dir, image_uri, HF_MODEL_ID, and model_data, using the reported code/inference.py and container logs as the reproduction path. Document whether custom inference is honored when image_uri and HF_MODEL_ID are used, and explain the supported setup and expected logging behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100