moby / moby/buildkit

ADD with an outdated checksum succeeds using outdated content, where a build error should occur

Open
#7,108 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
10.3k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
50

Description

Note: Created with the help of Claude AI

ADD --checksum with an outdated checksum can produce a build that appears to succeed while using
outdated file content. A build error should have occurred, because the content at the URL does not match
the declared checksum.

This happens when the URL is updated and the checksum is not updated with it. The URL is not part of the
cache key, so two URLs sharing a final path segment share a cache entry and the previously cached content
is reused. No request is made and nothing is compared.

Reproducer

https://github.com/k0pernikus/add-cached-file-even-after-content-change

Two files with the same basename in different directories, served over python -m http.server:

path content sha256
folder_1/same_file ALPHA 1921b918b15842c7fdb115078e610263fac85f159c1d8e0ecec3d89a0faa4005
folder_2/same_file BRAVO 8a51c1b8853b568b5ca25570ef461d6f5a8896f64a952364ed61be1a1e99eb00
# syntax=docker/dockerfile:1
FROM scratch
ARG SRC_URL
ARG SRC_SHA256
ADD --checksum=sha256:${SRC_SHA256} ${SRC_URL} /payload

Observed

build URL declared digest /payload expected
1 folder_1/same_file ALPHA's ALPHA ALPHA
2 folder_2/same_file ALPHA's (stale) ALPHA a failed build
3 folder_2/same_file BRAVO's BRAVO BRAVO

Build 2 logs the reused layer:

#6 [1/1] ADD --checksum=sha256:1921b918... http://127.0.0.1:8099/folder_2/same_file /payload
#6 CACHED

Build 3 shows that URL serves different bytes. Build 2's content therefore came from the cache, not the
server. The HTTP server's access log records no request during build 2.

Mechanism

In source/http/source.go, resolveMetadata calls resolveMetadataStatic, which returns early when a
checksum is declared, without making a request:

if hs.src.Checksum != "" {
    return &Metadata{
        Digest:   hs.src.Checksum,
        Filename: getFileName(hs.src.URL, hs.src.Filename, nil),
    }, nil
}

CacheKey then returns formatCacheKey(md.Filename, md.Digest, md.LastModified). The URL is not among
those inputs. getFileName reduces it to path.Base(u.Path), so folder_1/same_file and
folder_2/same_file both reduce to same_file. LastModified is nil because nothing was fetched.

Environment

Reproduced on two BuildKit versions, both with the docker driver embedded in dockerd:

  • GitHub Actions ubuntu-latest: BuildKit v0.20.2, Docker 28.0.4, buildx v0.19.0
  • Local: BuildKit v0.32.2, Docker 29.7.2, buildx v0.36.1

linux/amd64, frontend docker/dockerfile:1.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in source/http/source.go, focusing on resolveMetadata, resolveMetadataStatic, and CacheKey. Run the linked reproducer to observe the cache reuse and HTTP access logs. Done means changing the behavior so an outdated checksum cannot produce a successful build from cached content, while matching content still builds successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.