GoogleCloudPlatform / GoogleCloudPlatform/kubernetes-engine-samples
Update tfserve and torchserve example to include server metrics export
- Dominant language
- HCL
- Stars
- 1.4k
- Forks
- 1.3k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 4
Description
Currently GKE TF Serving example [ai-ml/gke-online-serving-single-gpu/src/gke-config/deployment-tfserve.yaml](https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/compare/main...chelseychen:kubernetes-engine-samples:main?expand=1#diff-1cec4fd55dbb3b8af73e87e5eb7809893ab458e992af2b2f7772c8d11d16c671) and GKE TorchServe example [ai-ml/t5-model-serving/model/config.properties](https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/compare/main...chelseychen:kubernetes-engine-samples:main?expand=1#diff-515c78592d12f200749b9255b8ddeb541b78d911975b3773ba2546ed5005b64e) don't support metrics export. Updating the example to include metrics export allows users who follow GKE public docs to deploy these model servers have ability to easily monitor their model servers.
These two examples are used in the following GKE public docs:
- https://cloud.google.com/kubernetes-engine/docs/tutorials/online-ml-inference#standard
- https://cloud.google.com/kubernetes-engine/docs/tutorials/scalable-ml-models-torchserve
Contributor guide
Assessment
This issue has not been assessed yet.