elastic / elastic/elastic-package

Propagate `builtPackageRoot` to all test runners and testers

Open
#3,530 1 comment 0 reactions 0 assignees View on GitHub
Team:Ecosystem
Dominant language
Go
Stars
72
Forks
141
Avg merge
19h 42m
Merged PRs (30d)
55

Description

## Summary

All test runners and testers currently receive only `packageRoot` (the source tree) and call
`builder.ReadBuiltPackageManifest()` ad-hoc when they need resolved composable input types.
This is inconsistent and incorrect for composable packages and packages using `.link` files,
where the Build Bundle is the only authoritative content source.

The fix is to:
1. Rename `PackageRoot` → `SourceRoot` across all options structs and internal fields.
2. Add `BuiltPackageRoot` alongside it.
3. Compute `builtPackageRoot` once in `cmd/testrunner.go` and propagate it through all runners.
4. Update each runner/tester to read **package content** (manifests, pipelines, fields, assets, sample events) from `builtPackageRoot` and **`_dev/` content** (test configs, deploy resources, ECS dependencies) from `sourceRoot`.

## Problem

`elastic-package test` introduced the Build Bundle for composable packages. The Build Bundle at
`build/packages///` differs from the source in three ways:

- `_dev/` is stripped
- `.link` stubs are replaced with materialized file content
- `package:` composable input references are resolved into concrete input types

Testers currently read everything from the source tree, then call `builder.ReadBuiltPackageManifest()`
ad-hoc in a few places. This means `.link` files in pipelines/fields are not resolved, and
composable package input types may not be correct.

## Rule

| What | Where to read |
|------|--------------|
| Manifests, pipelines, fields, assets, sample events | `builtPackageRoot` |
| Test case files, deploy configs, ECS deps (`_dev/build/build.yml`) | `sourceRoot` |
| Coverage report paths | `sourceRoot` (so reports map to developer-editable files) |

## Files to change

| File | Change |
|------|--------|
| `internal/testrunner/testrunner.go` | Rename `PackageRoot` → `SourceRoot`; add `BuiltPackageRoot` to `TestOptions` |
| `cmd/testrunner.go` | Compute `builtPackageRoot` via `builder.BuildPackagesDirectory(sourceRoot, "")`; pass both in all `TestOptions{}` literals |
| `internal/testrunner/runners/pipeline/runner.go` | Rename + add fields; manifest from built; pass both to tester |
| `internal/testrunner/runners/pipeline/tester.go` | Rename + add fields; `dataStreamRoot` computed from built; manifests from built; `fields.CreateValidator` keeps `sourceRoot` |
| `internal/testrunner/runners/pipeline/coverage.go` | Rename `options.PackageRoot` → `options.SourceRoot` only; no logic changes |
| `internal/testrunner/runners/system/runner.go` | Rename + add fields; manifest from built; pass both to tester |
| `internal/testrunner/runners/system/tester.go` | Rename + add fields; `r.dataStream` = built data stream root; service deployer call sites re-derive source on demand |
| `internal/testrunner/runners/static/runner.go` | Rename + add fields; manifest from built for `PackageHasDataStreams`; test folders from source |
| `internal/testrunner/runners/static/tester.go` | Rename + add fields; manifests/fields/OTel check from built; `fields.CreateValidator` keeps `sourceRoot` |
| `internal/testrunner/runners/asset/runner.go` | Rename + add fields; pass both to tester |
| `internal/testrunner/runners/asset/tester.go` | Rename + add fields; manifest and asset loading from built |
| `internal/testrunner/runners/policy/runner.go` | Rename + add fields; manifest from built; pass both to tester |
| `internal/testrunner/runners/policy/tester.go` | Rename + add fields; coverage paths stay `sourceRoot` |

**Not changed**: `internal/resources/fleetpolicy.go` — `FleetPackagePolicy.PackageRoot` stays source; it already resolves built internally via `builder.ReadBuiltPackageManifest()`.

## Key implementation notes

**`fields.CreateValidator` signature — keep `sourceRoot`**:
```go
fields.CreateValidator(repositoryRoot, sourceRoot, fieldsDir, ...)
// sourceRoot needed for _dev/build/build.yml (ECS BuildManifest)
// fieldsDir must point inside builtPackageRoot (materialized fields)
```

**Pipeline tester — preserve `!found` guard for input packages**:
```go
_, found, err := packages.FindDataStreamRootForPath(r.testFolder.Path)
if err != nil { ... }
if !found {
return nil, errors.New("data stream root not found") // preserves existing behavior
}
dataStreamRoot := filepath.Join(r.builtPackageRoot, "data_stream", r.testFolder.DataStream)
```

**System tester — `r.dataStream` becomes built root; service deployer needs source**:
```go
// Setup():
if r.testFolder.DataStream != "" {
r.dataStream = filepath.Join(r.builtPackageRoot, "data_stream", r.testFolder.DataStream)
}
// At service deployer call sites (need _dev/deploy/):
sourceDS, _, _ := packages.FindDataStreamRootForPath(r.testFolder.Path)
// pass DataStreamRoot: sourceDS
```

**Built data stream directory is NOT empty**: `BuildPackage()` copies all non-`_dev/` content, so
`build/packages///data_stream//` always contains `manifest.yml`,
`elasticsearch/ingest_pipeline/`, `routing_rules.yml`, and `fields/`. Test case files are read
from `testFolder.Path` (source `_dev/test/` paths) and are never derived from `dataStreamRoot`.

## Verification

1. `go build ./...` — no compile errors
2. `go test ./internal/testrunner/... ./internal/builder/...`
3. Test against a **non-composable** integration package — all test types must produce identical results (built ≈ source for these)
4. Test against a **composable** integration package — system test must resolve input types without ad-hoc `ReadBuiltPackageManifest` calls
5. Test against an **input package** — no data stream; `_dev/test//` paths still resolve correctly
6. After implementation, grep for remaining `builder.ReadBuiltPackageManifest` calls in `internal/testrunner/runners/` — there should be none

---

## Attachment: Domain Context

The full domain glossary and invariants for this refactor are in [CONTEXT.md](https://github.com/user-attachments/files/27470518/CONTEXT.md) at the
repository root. Key terms: Source Package, Build Bundle, Composable Package, Input Package,
`.link` File, `_dev/` Directory, `dataStreamRoot` invariant.

A detailed step-by-step implementation plan with per-file code-level instructions is in the
plan file (used during investigation):

[plan.md](https://github.com/user-attachments/files/27470570/plan.md)

— 10 steps
covering all 13 files.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.