feast-dev / feast-dev/feast

Docker images not published for versions 0.62.0 and 0.63.0 - Helm chart deployment fails with ImagePullBackOff

Open
#6,409 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
1d 21h
Merged PRs (30d)
15

Description

# Docker images not published for versions 0.62.0 and 0.63.0 - Helm chart deployment fails with ImagePullBackOff

## Labels
- `bug`
- `helm`
- `docker`
- `ci/cd`

## Issue Body

---

### Description

The Feast Helm charts for versions 0.62.0 and 0.63.0 reference Docker images that do not exist in the container registries, causing deployment failures with `ImagePullBackOff` errors.

### Affected Versions
- feast chart: 0.62.0
- feast chart: 0.63.0

### Missing Images

1. **Feature Server (Java)**
- Referenced by chart: `quay.io/feastdev/feature-server-java:0.62.0`
- Referenced by chart: `quay.io/feastdev/feature-server-java:0.63.0`
- Status: ❌ Does not exist in quay.io
- Alternative that works: `quay.io/feastdev/feature-server:0.62.0` (without `-java` suffix)

2. **Redis**
- Referenced by chart: `docker.io/bitnami/redis:5.0.7-debian-10-r32`
- Status: ❌ Old image removed from Bitnami registry

### Root Cause

GitHub Actions workflow for publishing Docker images failed:
**Failed Build**: https://github.com/feast-dev/feast/actions/runs/25299957140/job/74168167270

The build appears to have failed during the Docker image publishing step, resulting in the `feature-server-java` images never being pushed to the container registry.

### Steps to Reproduce

1. Add Feast Helm repository:
```bash
helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com
helm repo update
```

2. Install Feast chart:
```bash
helm install feast feast-charts/feast --version 0.63.0
```

3. Check pod status:
```bash
kubectl get pods -n feast
```

### Expected Behavior

Pods should start successfully with all images pulled from the registries.

### Actual Behavior

Pods fail with `ImagePullBackOff`:

```
NAME READY STATUS RESTARTS AGE
feast-feature-server-6f6cd9cbb6-r92lb 0/1 ErrImagePull 0 1m
feast-redis-master-0 0/1 ImagePullBackOff 0 1m
feast-redis-slave-0 0/1 ImagePullBackOff 0 1m
feast-transformation-service-86849757d5-vsnln 1/1 Running 0 1m
```

**Error messages:**
```
Failed to pull image "quay.io/feastdev/feature-server-java:0.63.0":
rpc error: code = NotFound desc = failed to pull and unpack image
"quay.io/feastdev/feature-server-java:0.63.0": failed to resolve reference
"quay.io/feastdev/feature-server-java:0.63.0":
quay.io/feastdev/feature-server-java:0.63.0: not found
```

```
Failed to pull image "docker.io/bitnami/redis:5.0.7-debian-10-r32":
rpc error: code = NotFound desc = failed to pull and unpack image
"docker.io/bitnami/redis:5.0.7-debian-10-r32": failed to resolve reference
"docker.io/bitnami/redis:5.0.7-debian-10-r32":
docker.io/bitnami/redis:5.0.7-debian-10-r32: not found
```

### Verification

Testing image availability:

```bash
# feature-server-java does NOT exist:
docker pull quay.io/feastdev/feature-server-java:0.63.0
# Error: manifest unknown: manifest unknown

# feature-server (without -java) DOES exist:
docker pull quay.io/feastdev/feature-server:0.63.0
# Success: image pulled

# Transformation server DOES exist:
docker pull quay.io/feastdev/feature-transformation-server:0.63.0
# Success: image pulled
```

### Environment

- Kubernetes Version: 1.28+
- Helm Version: 3.x
- Feast Chart Version: 0.62.0 / 0.63.0
- Container Runtime: containerd

### Impact

- **Severity**: High - Helm chart deployments are completely broken
- **Users Affected**: Anyone attempting to deploy Feast 0.62.0 or 0.63.0 via Helm
- **Workaround Exists**: Yes, but requires manual image overrides in values.yaml

### Suggested Fix

1. **Immediate**: Re-run the failed GitHub Actions workflow to publish missing images
2. **Short-term**: Update Helm charts to reference correct image names (without `-java` suffix)
3. **Long-term**:
- Add automated tests to verify published images exist before releasing charts
- Update Redis chart dependency to use current Bitnami Redis images
- Consider adding image verification to CI/CD pipeline

### Workaround

Users can override images in values.yaml:

```yaml
feature-server:
image:
repository: quay.io/feastdev/feature-server # without -java suffix
tag: "0.62.0"

redis:
image:
registry: docker.io
repository: bitnami/redis
tag: "7.2.4-debian-12-r9" # current stable version
```

### Related Issues

- Helm chart versions affected: 0.62.0, 0.63.0
- Failed CI/CD run: https://github.com/feast-dev/feast/actions/runs/25299957140/job/74168167270

### Additional Context

This issue blocks production deployments using the official Helm charts. The transformation service works because its image was successfully published, but the feature server (the core component) cannot start due to the missing image.

---

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.