moby / moby/buildkit

`COPY --rewrite-timestamp ...` to apply SOURCE_DATE_EPOCH build arg value to the timestamps of the files

Open
#6,348 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status/triage
Dominant language
Go
Stars
10.3k
Forks
1.5k
Avg merge
1d 23h
Merged PRs (30d)
48

Description

Description

Hi! Trying to produce reproducible layers, I realize that COPYing files from some layer into a scratch final stage does not respect SOURCE_DATE_EPOCH build arg.

For example, I'm having to do something like:

FROM ... AS some-stage
RUN --mount=from=some-previous-stage,dst=/bla ... >/this
ARG SOURCE_DATE_EPOCH
RUN find /this -print0 | xargs -0 touch --no-dereference --date=@$SOURCE_DATE_EPOCH

FROM scratch AS some-layer
COPY --link --from=some-stage /this /there

Note: I am using this "run --mount then copy --from" pattern many many times until a final scratch stage. Also, I am using --cache-from/to with type=registry,mode=max and --output=type=tar. This is buildx notation but you get the idea.

First: am I doing something wrong or unexpected? Checking with diffoci it seems I really do have to touch files to get a quiet diff.

I would like to extend on today's implementation of the build arg and the dockerfile frontend regarding rewrite-timestamp.
I propose that, given the build arg set, COPY should support a new flag which will touch all files added in this stage.

FROM ... AS some-stage
RUN --mount=from=some-previous-stage,dst=/bla ... >/this

FROM scratch AS some-layer
COPY --link --from=some-stage --rewrite-timestamp /this /there

Per https://github.com/moby/buildkit/blob/202e28fe031a3be7eba17fb4382e4bbb0acf69b3/docs/build-repro.md#source_date_epoch rewrite-timestamp only affects the image exporter.

cc https://github.com/moby/buildkit/pull/2911 https://github.com/moby/buildkit/issues/3973#issuecomment-3318117271

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the SOURCE_DATE_EPOCH and rewrite-timestamp discussion in docs/build-repro.md, then trace the Dockerfile frontend and image exporter mentioned in the issue. Determine how a new COPY flag should apply the build-arg timestamp to files added to a stage, and validate that the resulting layers are reproducible without the manual touch step.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, dockerfile, go
Domain
build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.