GoogleCloudPlatform / GoogleCloudPlatform/training-data-analyst

Distributed Training using GPUs on Cloud AI Platform - lab resource provisioning

Open
#2,284 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
8.6k
Forks
6.1k
Avg merge
4h 44m
Merged PRs (30d)
2

Description

Hello team,

There is a resource provisioning issue with [this lab](https://github.com/GoogleCloudPlatform/training-data-analyst/blob/51c9720aab7f7522deebf7ab73d6164278b94943/courses/machine_learning/deepdive2/production_ml/labs/distributed_training.ipynb)
When trying to do task#3 from the notebook, attach GPUs and rerun the training job I receive the below error.

`"error": {
"code": 429,
"message": "Quota failure for project qwiklabs-gcp-XXXXXXXX. The request for 2 K80 accelerators exceeds the allowed maximum of 0 A100, 0 K80, 0 P100, 0 P4, 0 T4, 0 TPU_V2, 0 TPU_V2_POD, 0 TPU_V3, 0 TPU_V3_POD, 0 V100 accelerators. To read more about Cloud ML Engine quota, see https://cloud.google.com/ml-engine/quotas.",
"status": "RESOURCE_EXHAUSTED",
"details": [
{
"@type": "type.googleapis.com/google.rpc.QuotaFailure",
"violations": [
{
"subject": "qwiklabs-gcp-00-XXXXXXXX",
"description": "The request for 2 K80 accelerators exceeds the allowed maximum of 0 A100, 0 K80, 0 P100, 0 P4, 0 T4, 0 TPU_V2, 0 TPU_V2_POD, 0 TPU_V3, 0 TPU_V3_POD, 0 V100 accelerators."`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.