elastic / elastic/integrations

backport_branch.sh: keep packages that are sources of .link files in the target package

Open
#20,726 1 comment 0 reactions 0 assignees View on GitHub
Team:Ecosystem
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 17h
Merged PRs (30d)
225

Description

## Problem

When creating a backport branch, the script removes all packages except the target package and any packages it declares under `requires.input` / `requires.content` in its `manifest.yml` (handled by `get_required_package_names` in `backport_branch_lib.sh`).

However, packages can contain [`.link` files](https://www.elastic.co/docs/extend/integrations/developer_workflow_links) — plain-text files with a `.link` extension managed by `elastic-package`. Each `.link` file contains a path (relative to the link file itself) pointing to a source file whose content it mirrors, plus a SHA-256 checksum. That relative path can traverse `../` boundaries and reference a file that lives inside a **different package**.

When a linked file's source package is not the target package and is not listed under `requires.*`, `remove_other_packages` will delete it, leaving the `.link` file pointing at a non-existent file and breaking the backport branch.

This was raised during review of [elastic/package-spec#1209](https://github.com/elastic/package-spec/pull/1209#discussion_r3782819412).

## Expected behaviour

`backport_branch.sh` should detect all `*.link` files inside the target package, resolve their source paths, identify which package each source file belongs to (by finding the nearest `manifest.yml` ancestor), and add those packages to `packages_to_keep` — the same list that already preserves `requires.*` dependencies.

`.github/CODEOWNERS` entries for those extra packages must also be retained, which is already handled generically by `remove_other_packages` for packages in `packages_to_keep`, so no extra logic is needed there once the package list is correct.

## Implementation hints

`backport_branch_lib.sh` already has the pattern to follow in `get_required_package_names`. A new function — e.g. `get_linked_source_package_names` — should:

1. Walk all `*.link` files under the given package path (`find "$package_path" -name "*.link"`).
2. Read the first whitespace-delimited field from each file (the relative path to the source).
3. Resolve the absolute path: `dirname()/`.
4. Walk up from that resolved path to find the nearest directory containing a `manifest.yml` — that directory is the source package root.
5. If the source package root differs from the target package root, emit its path so the caller can add it to `packages_to_keep`.

For reference, `elastic-package` implements the equivalent logic in Go at [`internal/files/linkedfiles.go`](https://github.com/elastic/elastic-package/blob/main/internal/files/linkedfiles.go) — specifically `listLinkedFiles` + `newLinkedFile` (which populates `IncludedPackageName`).

## Files to change

- `.buildkite/scripts/backport_branch_lib.sh` — add `get_linked_source_package_names`
- `.buildkite/scripts/backport_branch.sh` — call it alongside `get_required_package_names` when building `packages_to_keep`

Contributor guide

Open the contributing guide

Research direction

Start with .buildkite/scripts/backport_branch_lib.sh, especially get_required_package_names, and .buildkite/scripts/backport_branch.sh where packages_to_keep is built. Trace the existing package-removal flow and the .link path rules described in the issue. Done means a backport retains packages containing linked sources and their CODEOWNERS entries without changing unrelated package selection.

Written by the indexing model from the issue text.

Assessment

Tech stack
bash
Domain
build-system, release
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.