coder / coder/terraform-provider-envbuilder
envbuilder_cached_image outputs wrong digest — doesn't match what envbuilder actually pushes
- Dominant language
- Go
- Stars
- 13
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
I'm running into a digest mismatch between what `envbuilder_cached_image` computes and what envbuilder actually pushes to the cache registry.
**Setup:** envbuilder v1.3.0, provider v1.0.0, registry is CNCF Distribution v3 (`registry:3`), running on Kubernetes with Coder v2.30.3.
I have a template that uses the `exists` attribute to fall back to the builder image on cold cache:
```hcl
image = envbuilder_cached_image.cached[0].exists ? envbuilder_cached_image.cached[0].image : local.devcontainer_builder_image
```
First run works fine — `exists=false`, falls back to builder image, envbuilder builds the devcontainer and pushes to cache with `ENVBUILDER_PUSH_IMAGE=true`. All good.
On the second run the probe finds all cached layers, returns `exists=true`, and outputs something like:
```
registry.example.com/coder/cache@sha256:4778c7ff...
```
But the manifest that envbuilder actually pushed to the registry has a different digest:
```
sha256:a4ccf717...
```
Kubernetes tries to pull the provider's digest → `ImagePullBackOff` because that manifest simply doesn't exist in the registry.
I dug into this and the issue is that `runCacheProbe` doesn't read the manifest from the registry. It assembles a new `v1.Image` from the cached layers and computes `.Digest()` on that. The resulting manifest bytes are different from what envbuilder pushed — so different SHA256.
I checked the actual manifests and the layers are identical. The difference is in the manifest serialization itself. For example, the pushed manifest has `"mediaType":"application/vnd.oci.image.manifest.v1+json"` but the `Manifest` struct in go-containerregistry tags that field with `omitempty` — so if the probe doesn't set it, the field gets dropped from the JSON and the hash changes completely. The config digest also likely differs (different timestamps/history between the probe's fake build and the real build).
I think the fix would be for the provider to either:
- Read the actual manifest back from the registry (e.g. fetch the `latest` tag) after confirming layers exist, instead of computing the digest from a locally-assembled image
- Or have envbuilder store the digest somewhere the provider can read it back
Happy to provide more details if needed. For now I've had to disable the provider and rely on `ENVBUILDER_CACHE_REPO` env var for runtime caching, which works but doesn't give the fast-startup benefit of pulling a pre-built image.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing runCacheProbe and the provider's cached-image digest path using the envbuilder v1.3.0 setup described in the issue. Compare the computed digest with the manifest pushed to the registry, including media type and config metadata; done means the provider returns a pullable digest matching the pushed manifest.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes, terraform
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100