huggingface / huggingface/Microsoft-Azure
Include inference deployment examples to Azure Kubernetes Service
- Dominant language
- Makefile
- Stars
- 16
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
The Microsoft Azure Kubernetes Service (AKS) team has curated simple inference deployment examples using the open-source KAITO Sandbox project in the Cloud Native Computing Foundation (CNCF). We propose the following examples using KAITO to deploy Hugging Face LLMs onto an AKS cluster, in a dedicated `aks` directory in `huggingface/Microsoft-Azure/examples` and associated `aks` documentation in `huggingface/Microsoft-Azure/docs/source`. The examples are listed in:
* https://github.com/kaito-project/kaito/tree/main/examples/inference
* BYO models from HuggingFace can easily be deployed with https://github.com/kaito-project/kaito/blob/9e20ccec0b454ab9445ab206871b58c92e0ceae6/examples/custom-model-integration/reference-image-deployment.yaml and specifying the HuggingFace model identifier
Contributor guide
Assessment
This issue has not been assessed yet.