dotCMS / dotCMS/core

Release CI: fetch-depth: 0 checkouts cost ~19-20 min per job (fetch of all ~2k branches)

Open
#37,178 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Team : Platform
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Problem

Release runs spend most of their wall-clock time in git fetch, not building. From run 32759840677 (Release 26.08.24-01):

Job Total Git fetch
Prepare Release 22m04s ~20m33s
Build / Initial Artifact Build 34m22s ~19m13s

Root cause: actions/checkout@v4 with fetch-depth: 0 fetches all refs — every branch (~2,100+ currently on the remote) plus full history (~1.1 GB pack). GitHub has to negotiate a pack spanning every branch tip, which is why each fetch takes ~19-20 minutes. The build fetch log even shows it downloading branches unrelated to the release tag.

The release build path only needs:

  • Build phase: the tag snapshot (ref: v<version>). The only git consumer in the Maven build is buildnumber-maven-plugin (scmRevision), which reads HEAD only — works on a shallow clone.
  • Release prepare: HEAD of the default branch, plus (when explicitly supplied) the release commit SHA, which GitHub serves directly (uploadpack.allowAnySHA1InWant).

Measurements (cold clone, tag v26.08.24-01)

Method Time
Full-ref fetch (fetch-depth: 0) ~19 min in CI
Single-tag full-depth fetch 9m41s
Single-tag shallow fetch (--depth 1) 77s
Tarball snapshot (codeload) 24s, but loses .gitscmRevision falls back to UNKNOWN

Proposed fix

  • cicd_comp_build-phase.yml: ref-based checkout → fetch-depth: 1 (PR-check path that needs history stays at fetch-depth: 0)
  • cicd_comp_release-prepare-phase.yml: shallow checkout; explicit release commits fetched individually by SHA

Expected savings: ~38 of ~66 min per release run, and the same win in LTS, java-variant, manual-deploy, CLI, and nightly builds that pass a ref.

Related / follow-ups

  • Repo has ~2,100 branches of which ~1,800 are >6 months stale — backup + cleanup would further shrink ref advertisement
  • bump-min-sdk-version also paid the full-fetch tax (fixed in PR #37177)
  • Correction: earlier analysis claimed Initialize / Check Changed Files paid the full-fetch tax — it does not. For PRs, dorny/paths-filter@v3 uses the GitHub REST API (pulls/{n}/files) to get changed files (no git fetch; ~1s step), and its checkout has no fetch-depth so it is already shallow (default depth 1, ~13s). The 2m26s seen in the release run was runner-queue time, and release passes change-detection: 'disabled' so the checkout/paths-filter steps were skipped entirely. Git-based detection only happens for push/merge_group events where no PR exists to query.
  • maven-job action declares a needs-history input that is never consumed (vestigial)

Root cause: fetch-depth: 0 semantics were misread

We assumed fetch-depth: 0 meant "no history / snapshot only". It actually means unlimited depth for ALL refsactions/checkout fetches every branch tip on the remote (~2,100+ branches here), plus full history (~1.1 GB pack). With a repo this size that's ~19-20 minutes per job of pure ref negotiation and pack transfer. The correct "snapshot only" setting is fetch-depth: 1.

Full audit of fetch-depth across .github/ (post-PR #37177)

Workflow / location Git usage after checkout Status
cicd_comp_build-phase.yml (ref path) buildnumber-maven-plugin reads HEAD only ✅ PR #37177 → fetch-depth: 1
cicd_comp_build-phase.yml (no-ref path) git diff origin/main...HEAD needs merge base ⏸ intentionally kept 0 — needs rework (GitHub changed-files API or targeted two-ref fetch)
cicd_comp_release-prepare-phase.yml git log -1, reset --hard, branch push ✅ PR #37177 → fetch-depth: 1 + per-SHA fetch guard
cicd_comp_test-phase.yml (setup + 25 matrix jobs) none (maven-job action has no git usage; Sonar is a separate workflow) ✅ PR #37177 → fetch-depth: 1
cicd_comp_deployment-phase.yml none ✅ PR #37177 → fetch-depth: 1
cicd_6-release.yml (bump-min-sdk-version) git fetch origin main, checkout -B, commit, push (main only) ✅ PR #37177 → fetch-depth: 1
cicd_4-nightly.yml (setup) git log --before <midnight> on main (max ~19 commits/day on main) ✅ PR #37177 → fetch-depth: 100
ai_claude-backend-reviewer.yml agent runs git diff/log/show between base & head SHAs ⏸ intentionally kept 0 — optimization possible but risky for the AI workflow
legacy-release_maven-release-process.yml deprecated workflow, same pattern as release-prepare ⏸ untouched (deprecated)
publish_docs.yml ✅ already fetch-depth: 1

Guidance going forward

  • Default actions/checkout (no fetch-depth) is already shallow (1) — only add fetch-depth when you genuinely need history.
  • Never use fetch-depth: 0 as "get everything I need" — with 2k+ branches it fetches the entire remote every job.
  • For a bounded lookback (e.g. nightly's midnight commit), use fetch-depth: N sized to the lookback window.
  • When you need an arbitrary commit SHA, git fetch --depth=1 origin <sha> works on GitHub (uploadpack.allowAnySHA1InWant).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing .github workflows cicd_comp_build-phase.yml and cicd_comp_release-prepare-phase.yml, then compare the fetch-depth changes in PR #37177. Verify that ref-based builds use shallow checkouts, release commits are fetched individually, and workflows needing history retain an explicit justification.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, github-actions, yaml
Domain
build-system, ci-cd, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.