elastic / elastic/docs

Optimize Git Clone Strategy for Buildkite Documentation Builds

Open
#3,305 0 comments 0 reactions 0 assignees View on GitHub
docs-build
Dominant language
Ruby
Stars
265
Forks
347
Avg merge
3d 15h
Merged PRs (30d)
2

Description

## Problem

The current `.buildkite/scripts/build_pr.sh` script clones full repositories (e.g., `kibana`) when running documentation builds for pull requests. For large repositories, this results in excessive data transfer (>4GB) and increases build times, even though only the PR head and some history are required.

## Proposed Improvements

- Use `git fetch --depth=` (shallow fetch) to avoid cloning the complete history and tags.
- Consider partial clone (`--filter=blob:none`) for further bandwidth reduction where supported.
- Only fetch the PR head and target branch with sufficient depth to support `git diff` against the target.
- Avoid creating unnecessary local branches—consider detached checkouts of the PR head when a branch isn’t needed.
- Document and test minimum required depth for reliable diff/merge-base computation across supported repositories.

**Goal**: Reduce network usage and accelerate build times in CI, especially for very large repositories.

---

Related context: [Relevant snippet from the script](https://github.com/elastic/docs/blob/cadf7270f81415f2da1cd16941efdf355693b279/.buildkite/scripts/build_pr.sh#L54):

```shell
git fetch origin pull/$GITHUB_PR_NUMBER/head:pr_$GITHUB_PR_NUMBER &&
```

_See [related discussion](https://github.com/elastic/docs/issues/1821) for previous optimizations._

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.