ADD with an outdated checksum succeeds using outdated content, where a build error should occur
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 10.3k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 50
Description
Note: Created with the help of Claude AI
ADD --checksum with an outdated checksum can produce a build that appears to succeed while using
outdated file content. A build error should have occurred, because the content at the URL does not match
the declared checksum.
This happens when the URL is updated and the checksum is not updated with it. The URL is not part of the
cache key, so two URLs sharing a final path segment share a cache entry and the previously cached content
is reused. No request is made and nothing is compared.
Reproducer
https://github.com/k0pernikus/add-cached-file-even-after-content-change
reprofails on the stepBuild 2 should invalidate the cache and get file content BRAVO:
https://github.com/k0pernikus/add-cached-file-even-after-content-change/actions/runs/33623957687updated-checksumpasses, because it updates the URL and the checksum together:
https://github.com/k0pernikus/add-cached-file-even-after-content-change/actions/runs/33623957548
Two files with the same basename in different directories, served over python -m http.server:
| path | content | sha256 |
|---|---|---|
folder_1/same_file |
ALPHA |
1921b918b15842c7fdb115078e610263fac85f159c1d8e0ecec3d89a0faa4005 |
folder_2/same_file |
BRAVO |
8a51c1b8853b568b5ca25570ef461d6f5a8896f64a952364ed61be1a1e99eb00 |
# syntax=docker/dockerfile:1
FROM scratch
ARG SRC_URL
ARG SRC_SHA256
ADD --checksum=sha256:${SRC_SHA256} ${SRC_URL} /payload
Observed
| build | URL | declared digest | /payload |
expected |
|---|---|---|---|---|
| 1 | folder_1/same_file |
ALPHA's | ALPHA |
ALPHA |
| 2 | folder_2/same_file |
ALPHA's (stale) | ALPHA |
a failed build |
| 3 | folder_2/same_file |
BRAVO's | BRAVO |
BRAVO |
Build 2 logs the reused layer:
#6 [1/1] ADD --checksum=sha256:1921b918... http://127.0.0.1:8099/folder_2/same_file /payload
#6 CACHED
Build 3 shows that URL serves different bytes. Build 2's content therefore came from the cache, not the
server. The HTTP server's access log records no request during build 2.
Mechanism
In source/http/source.go, resolveMetadata calls resolveMetadataStatic, which returns early when a
checksum is declared, without making a request:
if hs.src.Checksum != "" {
return &Metadata{
Digest: hs.src.Checksum,
Filename: getFileName(hs.src.URL, hs.src.Filename, nil),
}, nil
}
CacheKey then returns formatCacheKey(md.Filename, md.Digest, md.LastModified). The URL is not among
those inputs. getFileName reduces it to path.Base(u.Path), so folder_1/same_file and
folder_2/same_file both reduce to same_file. LastModified is nil because nothing was fetched.
Environment
Reproduced on two BuildKit versions, both with the docker driver embedded in dockerd:
- GitHub Actions
ubuntu-latest: BuildKit v0.20.2, Docker 28.0.4, buildx v0.19.0 - Local: BuildKit v0.32.2, Docker 29.7.2, buildx v0.36.1
linux/amd64, frontend docker/dockerfile:1.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in source/http/source.go, focusing on resolveMetadata, resolveMetadataStatic, and CacheKey. Run the linked reproducer to observe the cache reuse and HTTP access logs. Done means changing the behavior so an outdated checksum cannot produce a successful build from cached content, while matching content still builds successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100