GoogleCloudPlatform / GoogleCloudPlatform/kubernetes-engine-samples

Proposal: Ray on TPU "get-started" example (ai-ml/gke-ray/tpu/get-started)

Open
#2,119 0 comments 0 reactions 2 assignees Claimed by @spencer-p View on GitHub
Dominant language
HCL
Stars
1.4k
Forks
1.3k
Avg merge
2d 2h
Merged PRs (30d)
4

Description

## Summary

A new end-to-end sample for the full LLM lifecycle on a single Cloud TPU v6e
slice with Ray on GKE: provision, serve, prepare data, and fine-tune. Proposed
path `ai-ml/gke-ray/tpu/get-started/`, alongside the existing `tpu/` samples. It
uses the Ray Operator add-on and a small, ungated model (Qwen3-4B-Instruct-2507)
on a single-host v6e `2x4` slice, Spot by default.

## Why

Today's `ai-ml/gke-ray` samples cover serving, training, and basic TPU jobs
separately. There is no single onboarding path that takes a developer through
serve, data, and train on one TPU slice with the Ray Operator add-on, vLLM-TPU,
and JAX post-training. This fills that gap.

## Layout

```
ai-ml/gke-ray/tpu/get-started/
├── README.md # index
├── cluster/ # Terraform: GKE + Ray Operator add-on + v6e slice + GCS + monitoring
├── serve/ # Ray Serve + vLLM, OpenAI-compatible API
├── data/ # Ray Data: DPO dataset prep + batch prediction
└── train/ # Ray Train + Tunix DPO + LoRA
```

Consumed in order, cluster first, and all four run on the same slice.

## Notes

Tested end-to-end on v6e `2x4`. Will ship with license headers, region tags, a CI
workflow that dry-runs Terraform and builds the images, and a CODEOWNERS entry.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.