elastic / elastic/elastic-package

Add retry logic for Elastic Package Registry (EPR) HTTP requests during build

Open
#3,756 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
72
Forks
141
Avg merge
19h 42m
Merged PRs (30d)
55

Description

## Summary

Integrations that declare `requires.input` dependencies cause `elastic-package build` to download input packages from EPR at build time. Each dependency triggers multiple registry HTTP requests (package zip + detached signature). Unlike the Kibana client, the registry client performs single-shot requests with no retry, so transient network or server errors can fail builds and CI pipelines that previously did not depend on external registry availability.

Build reliability will increasingly depend on EPR availability as more integrations adopt `requires.input`. Review feedback on https://github.com/elastic/elastic-package/pull/3753 asked whether `elastic-package build` should gain retry support, pointing at the integrations release pipeline as a primary consumer.

**Origin:** https://github.com/elastic/elastic-package/pull/3753#discussion_r3569874493

## Background

https://github.com/elastic/elastic-package/pull/3753 documents migrating integrations to `requires.input`. The migration guide notes that build time now fetches required input packages from the package registry:

> At build time, `elastic-package build` fetches required input packages from the package registry …

See https://github.com/elastic/elastic-package/blob/main/docs/howto/migrate_integration_required_input_dependency.md and https://github.com/elastic/integrations/pull/19719 for the reference migration.

## Investigation: does elastic-package retry EPR requests today?

**No.** The registry client does not use the existing retry helper.

| Component | Retry? | Notes |
|-----------|--------|-------|
| [`internal/registry/client.go`](https://github.com/elastic/elastic-package/blob/main/internal/registry/client.go) | No | Plain `http.Client`; `get()` performs a single `Do()` per request |
| [`internal/requiredinputs/requiredinputs.go`](https://github.com/elastic/elastic-package/blob/main/internal/requiredinputs/requiredinputs.go) | No | Calls `eprClient.DownloadPackage()` with no retry wrapper |
| [`internal/retry/http.go`](https://github.com/elastic/elastic-package/blob/main/internal/retry/http.go) | Yes (utility) | Used by Kibana client only (`RetryMax: 10`, wait 1–5s, retries 429/5xx/network errors) |
| [`internal/kibana/client.go`](https://github.com/elastic/elastic-package/blob/main/internal/kibana/client.go) | Yes | Wraps HTTP client via `retry.WrapHTTPClient` |

**EPR requests per required input dependency during build:**

1. `GET /epr/{name}/{name}-{version}.zip` — download package
2. `GET /epr/{name}/{name}-{version}.zip.sig` — download signature for verification

Packages with multiple `requires.input` entries multiply these calls. The same registry client is also used by `elastic-package install`, `test`, `benchmark`, and `requires update` when resolving input dependencies — not only the standalone `build` command.

## Investigation: `elastic-package build` usage in elastic/integrations

Survey of https://github.com/elastic/integrations (July 2026):

### Pipeline / script execution (1 call site)

| Location | Usage | Retry wrapper? |
|----------|-------|----------------|
| [`.buildkite/scripts/common.sh#L903-L911`](https://github.com/elastic/integrations/blob/main/.buildkite/scripts/common.sh#L903-L911) | `${ELASTIC_PACKAGE_BIN} build --zip` inside `build_zip_package()` | **No** |
| [`.buildkite/scripts/build_packages.sh#L40`](https://github.com/elastic/integrations/blob/main/.buildkite/scripts/build_packages.sh#L40) | Calls `build_zip_package` for each package during publish | **No** |

This path runs in the [publish pipeline](https://github.com/elastic/integrations/blob/main/.buildkite/pipeline.publish.yml) (step `build-packages`) on `main` and backport branches. Failures are reported per package but not retried.

Note: the same [`common.sh`](https://github.com/elastic/integrations/blob/main/.buildkite/scripts/common.sh#L259-L260) file **does** use a `retry` shell helper for compiling the elastic-package binary itself (`retry 5 go build -o "${ELASTIC_PACKAGE_BIN}" …`), but not for `elastic-package build`.

### Pipeline comments only (3 files)

- [`.buildkite/pipeline.yml#L20`](https://github.com/elastic/integrations/blob/main/.buildkite/pipeline.yml#L20) — comment about default license for build
- [`.buildkite/pipeline.serverless.yml#L19`](https://github.com/elastic/integrations/blob/main/.buildkite/pipeline.serverless.yml#L19) — same
- [`.buildkite/pipeline.publish.yml#L13`](https://github.com/elastic/integrations/blob/main/.buildkite/pipeline.publish.yml#L13) — same

### GitHub Actions (documentation hints, not executed)

- [`.github/workflows/docs-edit-automation.yml#L138`](https://github.com/elastic/integrations/blob/main/.github/workflows/docs-edit-automation.yml#L138) and [L185](https://github.com/elastic/integrations/blob/main/.github/workflows/docs-edit-automation.yml#L185) — echoes `elastic-package build` in PR comment text for contributors

### Documentation / developer guidance (~15 references)

Manual usage documented in [`docs/extend/`](https://github.com/elastic/integrations/tree/main/docs/extend) (`build-it.md`, `quick-start.md`, `elastic-package.md`, etc.) and [`.agents/skills/validate-integration-docs/SKILL.md`](https://github.com/elastic/integrations/blob/main/.agents/skills/validate-integration-docs/SKILL.md).

### `ELASTIC_PACKAGE_BIN` without `build`

Other CI scripts use `${ELASTIC_PACKAGE_BIN}` for `stack up/down`, `check`, `install`, `test`, `benchmark`, `links check`, etc. — those paths may also hit EPR indirectly via install/test build steps, but only one script invokes the `build` subcommand directly.

### Current `requires.input` adoption in integrations

Only **1** integration package currently declares `requires.input` ([`packages/elastic_package_registry/manifest.yml`](https://github.com/elastic/integrations/blob/main/packages/elastic_package_registry/manifest.yml)). Adoption is expected to grow following the new HOWTO and reference migration (https://github.com/elastic/integrations/pull/19719).

## Acceptance criteria

- [ ] Transient EPR failures during `elastic-package build` (e.g. 503, connection reset) do not fail the build when the registry recovers within a reasonable window.
- [ ] Permanent failures (e.g. 404, invalid package, TLS errors) still fail fast with clear error messages.
- [ ] Behavior is covered by automated tests.
- [ ] Other commands that download from EPR via the registry client benefit from the same resilience (if applicable to the chosen approach).

## Test plan

- [ ] Automated tests for recoverable vs non-recoverable EPR failure scenarios.
- [ ] Manual verification: build an integration with `requires.input` dependencies against EPR.
- [ ] Run project test and lint targets before merge.

## References

- Review comment: https://github.com/elastic/elastic-package/pull/3753#discussion_r3569874493
- Integrations build call site: https://github.com/elastic/integrations/blob/main/.buildkite/scripts/common.sh#L903-L911
- Registry client: https://github.com/elastic/elastic-package/blob/main/internal/registry/client.go
- Existing retry helper: https://github.com/elastic/elastic-package/blob/main/internal/retry/http.go
- Kibana client (uses retry today): https://github.com/elastic/elastic-package/blob/main/internal/kibana/client.go
- Required input resolver: https://github.com/elastic/elastic-package/blob/main/internal/requiredinputs/requiredinputs.go
- Build command: https://github.com/elastic/elastic-package/blob/main/cmd/build.go
- Migration HOWTO: https://github.com/elastic/elastic-package/blob/main/docs/howto/migrate_integration_required_input_dependency.md

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with internal/registry/client.go and internal/retry/http.go, then compare how internal/kibana/client.go wraps its HTTP client. Trace internal/requiredinputs/requiredinputs.go to confirm the EPR requests used during builds and related commands. Done means recoverable EPR failures retry, permanent failures remain clear and fast, and automated tests cover both cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.