nebari-dev / nebari-dev/rayserve-pack

Expose head/worker `volumes` and `volumeMounts` in values.yaml (for shared PVCs, etc.)

Open
#30 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: blocked ⛔️
Dominant language
Makefile
Stars
1
Forks
4
Avg merge
3d 14h
Merged PRs (30d)
5

Description

Summary

Expose generic volumes and volumeMounts at the head.* and worker.* values levels so operators can mount arbitrary Kubernetes volumes (PVCs, ConfigMaps, Secrets, emptyDir, hostPath, etc.) into the Ray head + worker containers without touching chart source.

The underlying RayService.spec.rayClusterConfig.headGroupSpec.template.spec and .workerGroupSpecs[].template.spec already accept any standard pod-spec volumes / volumeMounts — this is purely a matter of surfacing them through values.yaml.

Motivating use case

Sharing a large read-mostly dataset (or a model-weights cache) across the Ray head + all workers via a single RWX PVC. Concretely: mount an NFS/EFS/CephFS-backed PVC with accessModes: [ReadWriteMany] at /mnt/data on head + every worker so that:

  • Actors don't each pull multi-GB datasets from remote storage on startup
  • Multiple workers can read the same materialized dataset without duplication
  • Checkpoints or intermediate output written by one worker are visible to others

This is a first-class pattern for Ray-based ML workflows (both training-adjacent and inference-cache scenarios). Right now, users hitting it fall back to helm post-renderers, kustomize patches, or vendoring the chart — all of which lose the "values file as the single source of truth" property that makes GitOps deployments legible.

Current chart surface (for context)

chart/values.yaml exposes for head + worker: containerEnv, resources, tolerations, readinessProbe, livenessProbe, and (worker) autoscaling replicas. The only mount surface today is orgCABundle (added in #16), which is hard-coded to CA-bundle-shape: a ConfigMap → an initContainer → a specific mount path. Great for its purpose, not generalizable.

Proposed shape

head:
  # existing fields...
  volumes: []           # standard pod-spec volumes list
  volumeMounts: []      # standard container-spec volumeMounts, applied to the ray-head container

worker:
  # existing fields...
  volumes: []
  volumeMounts: []

Example override a caller would use for the shared-PVC case:

head:
  volumes:
    - name: shared-data
      persistentVolumeClaim:
        claimName: ray-shared-data
  volumeMounts:
    - name: shared-data
      mountPath: /mnt/data

worker:
  volumes:
    - name: shared-data
      persistentVolumeClaim:
        claimName: ray-shared-data
  volumeMounts:
    - name: shared-data
      mountPath: /mnt/data

Implementation

Follow the exact pattern PR #16 established for orgCABundle — the surface + composition mechanics are already in the chart:

  • Add named helpers in _helpers.tpl — e.g. nebari-rayserve.head.volumes, .head.volumeMounts, .worker.volumes, .worker.volumeMounts — that render the user-supplied lists (or empty when unset)
  • In chart/templates/rayservice.yaml, concat the user-supplied lists with the orgCABundle equivalents so both features compose without one clobbering the other:
{{- $headVolumes := concat (.Values.head.volumes | default list) (fromYamlArray (include "nebari-rayserve.orgCABundle.volumes" .)) }}
{{- $headVolumeMounts := concat (.Values.head.volumeMounts | default list) (fromYamlArray (include "nebari-rayserve.orgCABundle.volumeMounts" .)) }}

Same pattern as the existing head.containerEnv + orgCABundle env-var concat in #16. When head.volumes / worker.volumes are empty (default), render should be byte-identical to today.

Alternatives considered

  • Post-render / kustomize patches — works today but hides config from the ArgoCD Application spec; painful to maintain.
  • Fork the chart — worst maintenance profile; diverges from upstream forever.
  • Continue treating orgCABundle as the only supported mount surface — doesn't cover the common ML workload case of a shared dataset/weights PVC.

Acceptance

  • head.volumes / head.volumeMounts and worker.volumes / worker.volumeMounts accepted as standard pod-spec / container-spec shapes
  • Empty defaults ([]) produce byte-identical render vs. current output — verifiable via helm template diff
  • Coexists with orgCABundle — enabling both simultaneously merges the volumes + mounts additively, not clobbering
  • README section documenting the new fields with the shared-PVC example above

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with chart/values.yaml, _helpers.tpl, and chart/templates/rayservice.yaml, following the existing orgCABundle and containerEnv composition from PR #16. Use helm template to verify empty defaults are byte-identical and that user volumes and mounts coexist with orgCABundle. Update the README with the new fields and shared-PVC example.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes, yaml
Domain
devops, documentation, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.