NVIDIA / NVIDIA/OpenShell

feat(testing): support independent gateway and supervisor image overrides

Open
#3,315 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

state:triage-needed
Dominant language
Rust
Stars
8.7k
Forks
1.3k
Avg merge
2d 11h
Merged PRs (30d)
253

Description

User Story

As an OpenShell developer or downstream integrator, I want to configure the gateway and supervisor test images independently, so that I can run OpenShell’s test wrappers against custom, forked, mirrored, or differently named images.

Problem Statement

OpenShell’s test wrappers generally derive gateway and supervisor images from a shared registry prefix and image tag:

${REGISTRY_VALUE}/gateway:${IMAGE_TAG}
${REGISTRY_VALUE}/supervisor:${IMAGE_TAG}

This assumes that both images use the same registry, use the standard gateway and supervisor repository names, and use the same tag.

That assumption is hard-coded across the Kubernetes, Docker, and Podman test setup paths. It prevents testing images such as:

registry.example.com/custom/openshell-gateway:branch-build
registry.example.com/custom/openshell-sandbox:branch-build

or images that are pinned independently by tag or digest.

Impact / Why This Matters

Users testing downstream builds, private registries, forks, rebuilt artifacts, or alternate image layouts must currently retag their images to match OpenShell’s expected repository names or modify the test scripts locally.

This makes test execution harder to reproduce and creates unnecessary friction for CI systems where gateway and supervisor images may be published under different names or built by different pipelines.

The current Docker and Podman wrappers already support some supervisor-specific overrides, but the behavior is inconsistent across wrappers and there is no equivalent general gateway-image override.

Proposed Design

Add independent gateway and supervisor image overrides to all test wrappers that select or construct OpenShell images.

For example:

GATEWAY_IMAGE="registry.example.com/custom/gateway:test"
SUPERVISOR_IMAGE="registry.example.com/custom/supervisor:test"

When the overrides are not set, preserve the existing registry-and-tag behavior:

GATEWAY_IMAGE="${GATEWAY_IMAGE:-${REGISTRY_VALUE}/gateway:${IMAGE_TAG_VALUE}}"
SUPERVISOR_IMAGE="${SUPERVISOR_IMAGE:-${REGISTRY_VALUE}/supervisor:${IMAGE_TAG_VALUE}}"

The resolved full image references should be used consistently across:

  • e2e/with-kube-gateway.sh;
  • e2e/with-docker-gateway.sh;
  • e2e/with-podman-gateway.sh;
  • other test setup wrappers that build, select, or deploy gateway and supervisor images;
  • Helm chart values;
  • local Docker or Podman image builds;
  • k3d and kind image imports;
  • external-driver image composition;
  • generated gateway configuration; and
  • diagnostic output.

The implementation should support both tagged and digest-pinned references.

Existing variables such as OPENSHELL_SUPERVISOR_IMAGE and OPENSHELL_DOCKER_SUPERVISOR_IMAGE must remain backward compatible. The implementation should either preserve them as aliases or define clear precedence between existing variables and the new common overrides.

A possible precedence order is:

  1. wrapper-specific existing override;
  2. common SUPERVISOR_IMAGE or GATEWAY_IMAGE override;
  3. registry-plus-tag fallback.

Alternatively, the project could use namespaced variables such as OPENSHELL_E2E_GATEWAY_IMAGE and OPENSHELL_E2E_SUPERVISOR_IMAGE to avoid collisions with unrelated tooling. If the simpler names are chosen, the documentation should clearly define their scope as test-wrapper inputs.

Acceptance Criteria

  • All relevant OpenShell test wrappers support independent gateway and supervisor image selection.
  • A custom gateway image can use a different repository, tag, or digest from the supervisor image.
  • Existing workflows using only the registry prefix and image tag continue to resolve the same images as before.
  • Kubernetes Helm deployments use the resolved gateway and supervisor image references.
  • Local Kubernetes image builds and k3d/kind image loading use the resolved image references.
  • Docker and Podman wrappers use the resolved image references consistently.
  • Existing Docker and Podman supervisor override variables remain compatible.
  • Full image tags and digest references are supported.
  • Wrapper output identifies the resolved gateway and supervisor images.
  • TESTING.md documents the new overrides, fallback behavior, and precedence rules.
  • Tests or shell validation cover the default fallback and independent override cases.

Alternatives Considered

Continue requiring image retagging

Users could continue retagging custom images as ${REGISTRY_VALUE}/gateway:${IMAGE_TAG} and ${REGISTRY_VALUE}/supervisor:${IMAGE_TAG}. This avoids changing the wrappers but is inconvenient, obscures the images actually under test, and is difficult to use with immutable or externally managed images.

Add separate registry prefixes

Separate gateway and supervisor registry variables could be added while retaining a shared tag. This would help with different registries but would not support independently tagged or digest-pinned images.

Use additional Helm values files

For Kubernetes, users could provide Helm overrides for the gateway and supervisor images. This would not address Docker or Podman wrappers, local image builds, image imports, or generated gateway configuration.

Related Issues

  • Issue #2067 discusses gateway and supervisor version compatibility, but does not address test-wrapper image overrides.
  • Issue #1299 discusses Kubernetes supervisor image delivery, but focuses on image-volume sideloading.
  • Issue #1597 discusses Kubernetes E2E harness behavior, but not custom gateway and supervisor image selection.

Agent Investigation

The current repository investigation found:

  • e2e/with-kube-gateway.sh derives both images from a shared registry and tag in Helm installation, local builds, k3d imports, and kind image loading.
  • The Kubernetes wrapper also embeds the supervisor image when building the external Kubernetes driver gateway image.
  • e2e/with-docker-gateway.sh supports OPENSHELL_DOCKER_SUPERVISOR_IMAGE and OPENSHELL_SUPERVISOR_IMAGE, but has no equivalent gateway-image override.
  • e2e/with-podman-gateway.sh supports OPENSHELL_SUPERVISOR_IMAGE, but does not provide a common gateway-image override.
  • Current CI workflows rely on the registry-plus-tag fallback, so preserving that fallback is required for backward compatibility.
  • The repository documentation currently describes OPENSHELL_REGISTRY and IMAGE_TAG as controlling the gateway/supervisor image pair.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading e2e/with-kube-gateway.sh, e2e/with-docker-gateway.sh, and e2e/with-podman-gateway.sh to trace how gateway and supervisor images flow through Helm, local builds, and image imports. Then inspect TESTING.md and existing supervisor override handling, preserving the registry-plus-tag fallback and defining precedence. Done means independent tagged or digest references work across the wrappers, diagnostics show the resolved images, and fallback and override cases are validated.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, helm, kubernetes, shell
Domain
devops, documentation, infrastructure, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.