aws / aws/sagemaker-pytorch-inference-toolkit
Documentation for inference.py `transform_fn`
- Dominant language
- Python
- Stars
- 143
- Forks
- 73
- PR merge metrics
- No merged PRs in 30d
Description
**What did you find confusing? Please describe.**
Huggingface have [documented](https://huggingface.co/docs/sagemaker/inference) how to use the sagemaker pytorch inference API in order to host their models. They make it quite clear that you must supply `model_fn` and then either `transform_fn` or (`input_fn`, `predict_fn` and `output_fn`). By using `transform_fn` you can have fine control of batch size for example, allowing you to handle large requests (in particular I have an issue where my batch transform jobs continuously die because the minimum payload of 1MB is way to large for my model - due to the large intermediate matrices I..e probabilities = batch_szie x num_labels)
I cannot find any mention of `transform_fn` in the documentation - https://sagemaker.readthedocs.io/en/stable/frameworks/pytorch/using_pytorch.html
It is mentioned in passing in one of the examples - https://sagemaker-examples.readthedocs.io/en/latest/frameworks/pytorch/get_started_mnist_deploy.html
**Describe how documentation can be improved**
Document the use of `transform_fn` as an alternative to `input_fn`, `predict_fn` and `output_fn`
**Additional context**
[Add any other context or screenshots about the documentation request here.]
This is how I was aware of `transform_fn`:
https://aws.amazon.com/blogs/machine-learning/run-computer-vision-inference-on-large-videos-with-amazon-sagemaker-asynchronous-endpoints/
The I found this:
(https://huggingface.co/docs/sagemaker/inference)
Contributor guide
Research direction
Start with the SageMaker PyTorch documentation at using_pytorch.html and compare its inference API coverage with the linked Hugging Face inference documentation and MNIST example. Document transform_fn as an alternative to input_fn, predict_fn, and output_fn, and consider the existing inference.py context; done means users can find and understand how to use transform_fn.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100