GoogleCloudPlatform / GoogleCloudPlatform/kubernetes-engine-samples

Update tfserve and torchserve example to include server metrics export

Open
#1,622 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
HCL
Stars
1.4k
Forks
1.3k
Avg merge
2d 2h
Merged PRs (30d)
4

Description

Currently GKE TF Serving example [ai-ml/gke-online-serving-single-gpu/src/gke-config/deployment-tfserve.yaml](https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/compare/main...chelseychen:kubernetes-engine-samples:main?expand=1#diff-1cec4fd55dbb3b8af73e87e5eb7809893ab458e992af2b2f7772c8d11d16c671) and GKE TorchServe example [ai-ml/t5-model-serving/model/config.properties](https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/compare/main...chelseychen:kubernetes-engine-samples:main?expand=1#diff-515c78592d12f200749b9255b8ddeb541b78d911975b3773ba2546ed5005b64e) don't support metrics export. Updating the example to include metrics export allows users who follow GKE public docs to deploy these model servers have ability to easily monitor their model servers.

These two examples are used in the following GKE public docs:
- https://cloud.google.com/kubernetes-engine/docs/tutorials/online-ml-inference#standard
- https://cloud.google.com/kubernetes-engine/docs/tutorials/scalable-ml-models-torchserve

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.