GoogleCloudPlatform / GoogleCloudPlatform/pubsec-declarative-toolkit

GCE support for G2 g2-standard-48 VM as wrapper of four L4 GPUs (CUDA CC 8.9 80G vram) via the LZ as raw headless GCE in addition to marketplace nvidia-rtx-virtual-workstation

Open
#655 4 comments 0 reactions 1 assignee Claimed by @fmichaelobrien View on GitHub
canary gpu workload
Dominant language
Shell
Stars
36
Forks
26
PR merge metrics
No merged PRs in 30d

Description

add example workload config for G2 in general - for cuda/tensorflow/keras/llm training/inference
https://cloud.google.com/blog/products/compute/introducing-g2-vms-with-nvidia-l4-gpus

Alternate NVidia workstation deployment is already working via marketplace
https://console.cloud.google.com/marketplace/product/nvidia/nvidia-rtx-virtual-workstation-windows-server-2022

## L4 GPUs per G2 VM
- https://cloud.google.com/compute/docs/gpus#l4-gpus
- 8 L4 per A2-standard-96 or 4 L4 per standard-48
- This requires the following additional quota requests (3)
- NVIDIA L4 GPUs from 4 to 8
- CPUs from 64 to 96 (includes CPUs (all regions)

```
| NAME | DIMENSIONS | REGION | REQUESTED LIMIT | APPROVED LIMIT |
+------------------+------------+--------+-----------------+----------------+
| CPUS_ALL_REGIONS | | GLOBAL | 96 | 96 |

```
- choose CUDA 12.1 over 11.8
image

```
gcloud compute instances create l4-4b --project=cuda-old --zone=us-east4-c --machine-type=g2-standard-48 --network-interface=network-tier=PREMIUM,stack-type=IPV4_ONLY,subnet=default --maintenance-policy=TERMINATE --provisioning-model=STANDARD --service-account=196717963363-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/cloud-platform --accelerator=count=4,type=nvidia-l4 --tags=http-server,https-server --create-disk=auto-delete=yes,boot=yes,device-name=l4-4b,image=projects/ml-images/global/images/c0-deeplearning-common-cu121-v20231105-debian-11,mode=rw,size=50,type=projects/cuda-old/zones/us-east4-c/diskTypes/pd-balanced --no-shielded-secure-boot --shielded-vtpm --shielded-integrity-monitoring --labels=goog-ec-src=vm_add-gcloud --reservation-affinity=any

(base) michael@l4-4b:~$ nvidia-smi
Fri Dec 1 01:42:34 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.86.10 Driver Version: 535.86.10 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA L4 Off | 00000000:00:03.0 Off | 0 |
| N/A 58C P0 29W / 72W | 4MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA L4 Off | 00000000:00:04.0 Off | 0 |
| N/A 60C P0 31W / 72W | 4MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 2 NVIDIA L4 Off | 00000000:00:05.0 Off | 0 |
| N/A 58C P0 31W / 72W | 4MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 3 NVIDIA L4 Off | 00000000:00:06.0 Off | 0 |
| N/A 58C P0 29W / 72W | 4MiB / 23034MiB | 4% Default |
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.