GoogleCloudPlatform / GoogleCloudPlatform/pubsec-declarative-toolkit
GCE support for G2 g2-standard-48 VM as wrapper of four L4 GPUs (CUDA CC 8.9 80G vram) via the LZ as raw headless GCE in addition to marketplace nvidia-rtx-virtual-workstation
- Dominant language
- Shell
- Stars
- 36
- Forks
- 26
- PR merge metrics
- No merged PRs in 30d
Description
add example workload config for G2 in general - for cuda/tensorflow/keras/llm training/inference
https://cloud.google.com/blog/products/compute/introducing-g2-vms-with-nvidia-l4-gpus
Alternate NVidia workstation deployment is already working via marketplace
https://console.cloud.google.com/marketplace/product/nvidia/nvidia-rtx-virtual-workstation-windows-server-2022
## L4 GPUs per G2 VM
- https://cloud.google.com/compute/docs/gpus#l4-gpus
- 8 L4 per A2-standard-96 or 4 L4 per standard-48
- This requires the following additional quota requests (3)
- NVIDIA L4 GPUs from 4 to 8
- CPUs from 64 to 96 (includes CPUs (all regions)
```
| NAME | DIMENSIONS | REGION | REQUESTED LIMIT | APPROVED LIMIT |
+------------------+------------+--------+-----------------+----------------+
| CPUS_ALL_REGIONS | | GLOBAL | 96 | 96 |
```
- choose CUDA 12.1 over 11.8
```
gcloud compute instances create l4-4b --project=cuda-old --zone=us-east4-c --machine-type=g2-standard-48 --network-interface=network-tier=PREMIUM,stack-type=IPV4_ONLY,subnet=default --maintenance-policy=TERMINATE --provisioning-model=STANDARD --service-account=196717963363-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/cloud-platform --accelerator=count=4,type=nvidia-l4 --tags=http-server,https-server --create-disk=auto-delete=yes,boot=yes,device-name=l4-4b,image=projects/ml-images/global/images/c0-deeplearning-common-cu121-v20231105-debian-11,mode=rw,size=50,type=projects/cuda-old/zones/us-east4-c/diskTypes/pd-balanced --no-shielded-secure-boot --shielded-vtpm --shielded-integrity-monitoring --labels=goog-ec-src=vm_add-gcloud --reservation-affinity=any
(base) michael@l4-4b:~$ nvidia-smi
Fri Dec 1 01:42:34 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.86.10 Driver Version: 535.86.10 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA L4 Off | 00000000:00:03.0 Off | 0 |
| N/A 58C P0 29W / 72W | 4MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA L4 Off | 00000000:00:04.0 Off | 0 |
| N/A 60C P0 31W / 72W | 4MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 2 NVIDIA L4 Off | 00000000:00:05.0 Off | 0 |
| N/A 58C P0 31W / 72W | 4MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 3 NVIDIA L4 Off | 00000000:00:06.0 Off | 0 |
| N/A 58C P0 29W / 72W | 4MiB / 23034MiB | 4% Default |
```
Contributor guide
Assessment
This issue has not been assessed yet.