coder / coder/terraform-provider-envbuilder

envbuilder_cached_image outputs wrong digest — doesn't match what envbuilder actually pushes

Open
#124 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
13
Forks
6
PR merge metrics
No merged PRs in 30d

Description

I'm running into a digest mismatch between what `envbuilder_cached_image` computes and what envbuilder actually pushes to the cache registry.

**Setup:** envbuilder v1.3.0, provider v1.0.0, registry is CNCF Distribution v3 (`registry:3`), running on Kubernetes with Coder v2.30.3.

I have a template that uses the `exists` attribute to fall back to the builder image on cold cache:

```hcl
image = envbuilder_cached_image.cached[0].exists ? envbuilder_cached_image.cached[0].image : local.devcontainer_builder_image
```

First run works fine — `exists=false`, falls back to builder image, envbuilder builds the devcontainer and pushes to cache with `ENVBUILDER_PUSH_IMAGE=true`. All good.

On the second run the probe finds all cached layers, returns `exists=true`, and outputs something like:
```
registry.example.com/coder/cache@sha256:4778c7ff...
```

But the manifest that envbuilder actually pushed to the registry has a different digest:
```
sha256:a4ccf717...
```

Kubernetes tries to pull the provider's digest → `ImagePullBackOff` because that manifest simply doesn't exist in the registry.

I dug into this and the issue is that `runCacheProbe` doesn't read the manifest from the registry. It assembles a new `v1.Image` from the cached layers and computes `.Digest()` on that. The resulting manifest bytes are different from what envbuilder pushed — so different SHA256.

I checked the actual manifests and the layers are identical. The difference is in the manifest serialization itself. For example, the pushed manifest has `"mediaType":"application/vnd.oci.image.manifest.v1+json"` but the `Manifest` struct in go-containerregistry tags that field with `omitempty` — so if the probe doesn't set it, the field gets dropped from the JSON and the hash changes completely. The config digest also likely differs (different timestamps/history between the probe's fake build and the real build).

I think the fix would be for the provider to either:
- Read the actual manifest back from the registry (e.g. fetch the `latest` tag) after confirming layers exist, instead of computing the digest from a locally-assembled image
- Or have envbuilder store the digest somewhere the provider can read it back

Happy to provide more details if needed. For now I've had to disable the provider and rely on `ENVBUILDER_CACHE_REPO` env var for runtime caching, which works but doesn't give the fast-startup benefit of pulling a pre-built image.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing runCacheProbe and the provider's cached-image digest path using the envbuilder v1.3.0 setup described in the issue. Compare the computed digest with the manifest pushed to the registry, including media type and config metadata; done means the provider returns a pullable digest matching the pushed manifest.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes, terraform
Domain
infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.