containerd / containerd/nerdctl

[CI]: git clone revamp, part II

Open
#4,031 0 comments 0 reactions 0 assignees View on GitHub
kind/feature
Dominant language
Go
Stars
10.4k
Forks
826
Avg merge
1d 23h
Merged PRs (30d)
44

Description

### What is the problem you're trying to solve

This is a second breakout from #4021.

1. git clones are currently done for every target platform. There is no reason for that, and clone could (should) be done only once per run, at maximum.

2. The same goes for go modules.
They are currently retrieved at build time - which means they are retrieved as many times as we build (per architecture, per version).
They could (should) instead be retrieved once, at clone time, using `go mod vendor`.

3. Also, go mod could (should) use a caching mount for the download cache location, to avoid redownloading modules that have been fetched already (eg: by another project), and also avoid putting every independent stage go mod cache into buildkit cache

4. Finally, it is convenient to be able to override the git repository location of a given project (be it because of maintenance or to test a fork). I recently faced exactly that problem (I am building libslirp, but they went offline for maintenance for a week).

The proposal here is to address these issues.

### Describe the solution you'd like

The updated form of the Dockerfile would look something like (pseudo code, should be very close to working):

```dockerfile

FROM --platform=$BUILDPLATFORM builder AS download-bypass4netns
ARG BYPASS4NETNS_VERSION
ARG BYPASS4NETNS_REPO
RUN --mount=target=/root/go/pkg/mod,type=cache \
git clone --quiet --depth 1 --branch "${BYPASS4NETNS_VERSION%@*}" https://"$BYPASS4NETNS_REPO".git . && \
git-checkout-tag-with-hash.sh "$VERSION" && \
go mod vendor

FROM --platform=$BUILDPLATFORM builder AS build-bypass4netnsd
ARG TARGETARCH
RUN --mount=from=download-bypass4netns,type=bind,target=/src,source=/src,rw \
exec 42>.lock; flock -x 42; \
GOPROXY=off GOFLAGS=--mod=vendor make blablbabla; \
flock -u 42
```

Let me know your thoughts on this to ~infzasx~ (cat is typing :-)) to inform upcoming PR.

### Additional context

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.