apache / apache/hugegraph

perf(docker): track follow-up image build optimizations

Open
#3,179 0 comments 0 reactions 0 assignees View on GitHub
ci-cd improvement
Dominant language
Java
Stars
3.2k
Forks
636
Avg merge
3d 11h
Merged PRs (30d)
14

Description

## Background

The shared Buildx Bake graph from apache/hugegraph#3171 and hugegraph/actions#27 removed four repeated Maven reactor builds. A real `latest` publication completed successfully with all four images, both `linux/amd64` and `linux/arm64`, the Compose/Gremlin checks, and the standalone Server smoke test.

Production run: https://github.com/hugegraph/actions/actions/runs/33302286614

The candidate build took 614 seconds. The main critical-path work was:

| Work | Observed duration |
| --- | ---: |
| Full Maven reactor build | about 285s |
| ARM64 runtime package installation under QEMU | about 149–171s |
| Local multi-platform image export | up to about 53s |
| Per-image registry cache export | up to about 100s |

The shared build-cache export took about 237 seconds but overlapped much of the runtime work, so BuildKit vertex durations must not be added directly.

## Follow-up candidates

### 1. Put stable runtime dependency layers before source-dependent artifacts

The four runtime stages currently copy Maven artifacts before running `apt-get`. Any source change therefore invalidates package installation as well, which is particularly expensive under ARM64 emulation.

Move stable package installation before `COPY --from=build`. Where a command also edits a copied configuration file, split the stable package installation from the source-dependent configuration edit.

Keep the existing operational tools, including `vim`, `procps`, `curl`, `dumb-init`, and `iproute2` where applicable. Removing commonly used operational tools is explicitly out of scope.

Expected result: reuse the runtime dependency layer on ordinary source changes and reduce total build time by roughly 1–2 minutes, subject to a real benchmark.

### 2. Evaluate a module-scoped Maven build

Determine whether the image build can request only the Server, PD, and Store distribution modules plus their reactor dependencies instead of installing all 38 modules.

Do not skip required assembly inputs or silently weaken validation. Compare the produced distributions and image contents before adopting this change.

### 3. Re-evaluate native ARM builds only if the simpler changes are insufficient

A native ARM runner could remove QEMU runtime work, but it would add runner coordination and manifest assembly complexity. Keep the current single-job Bake flow unless measurements show a clear end-to-end benefit without reintroducing temporary architecture tags or partially published images.

### 4. Measure cache export separately

Registry cache export is a meaningful part of the build step, but removing it can merely shift time to the final image push or the next build. Compare both the candidate-build duration and the complete publish duration; do not optimize one metric by moving the same work elsewhere.

## Acceptance criteria

- Preserve direct Dockerfile builds and the shared `docker/bake.hcl` flow.
- Preserve the current runtime tools and published image behavior.
- Produce the same four final image tags with both amd64 and arm64 variants.
- Keep the existing local multi-platform inspection, Compose/Gremlin checks, and standalone Server smoke test before publication.
- Benchmark on the same runner class with a real registry cache and publish path.
- Report both the candidate-build duration and the complete job duration before and after.
- Prefer the runtime-layer reorder first; pursue the other candidates only when their measured benefit justifies the added complexity.

Contributor guide

Open the contributing guide

Research direction

Start with docker/bake.hcl and locate the four runtime stages and their Dockerfiles, then reproduce the production build with registry cache and multi-platform publishing. Compare the runtime-layer reorder first, while checking module-scoped builds and cache export only when justified. Done means the same four amd64/arm64 image tags, checks, smoke test, and before-and-after candidate and complete-job timings are preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, dockerfile, java
Domain
build-system, devops, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.