elastic / elastic/elastic-package
README template functions {{ inputDocs }} and {{ fields }} produce incomplete output for composable (input-package) integrations
- Dominant language
- Go
- Stars
- 72
- Forks
- 141
- Avg merge
- 19h 42m
- Merged PRs (30d)
- 55
Description
## Summary
`{{ inputDocs }}` and `{{ fields }}` README template functions produce incomplete output for integrations that use composable (input-package) packages. `{{ inputDocs }}` always renders empty, and `{{ fields }}` silently omits any fields contributed by the required input package.
## Background
Composable integrations declare their inputs via `streams[].package: ` in data stream manifests instead of the conventional `streams[].input`. During `elastic-package build`, `RequiredInputsResolver.Bundle` rewrites those references to `streams[].input` and copies field definitions from the input package into the data stream's `fields/` directory — but both operations happen exclusively in the **build root** (`build/packages///`). The source tree is never modified.
`UpdateReadmes` is called after `Bundle` and correctly receives `buildPackageRoot`, but it passes only `packageRoot` (source) to `GenerateReadme` → `renderReadme`, where all template closures read their data. Because the composable-specific data never reaches the source tree, both affected functions return incomplete results.
## Affected template functions
| Template function | What it reads | What it misses |
|---|---|---|
| `{{ inputDocs }}` | `data_stream/*/manifest.yml` from source for `streams[].input` | `streams[].input` — absent in source; only written to build root by `resolveStreamInputTypes` |
| `{{ fields }}` | `data_stream/*/fields/*.yml` from source | `-fields.yml` — written only to build root by `bundleDataStreamFields` |
Unaffected: `{{ ilm }}`, `{{ transform }}`, `{{ event }}`, `{{ alertRuleTemplates }}`, `{{ sloTemplates }}`.
## Observed behaviour
- `{{ inputDocs }}` renders as an empty string for any package that uses `streams[].package`.
- `{{ fields }}` omits fields that originate from the required input package (defined in the input package's `fields/` directory and not overridden in the composable integration).
Discovered while working on https://github.com/elastic/integrations/pull/19719 (`elastic_package_registry`, which uses `streams[].package: prometheus_input`).
## Expected behaviour
Both `{{ inputDocs }}` and `{{ fields }}` should produce the same output that a conventional (non-composable) integration with equivalent resolved manifests and fields would produce.
## Root cause
`internal/docs/readme.go` — `renderReadme` registers all template closures over `packageRoot` (source path). `buildPackageRoot` is available at the `UpdateReadmes` call site (`internal/builder/packages.go:259`) but is not threaded into `GenerateReadme` or `renderReadme`.
Relevant code:
- `internal/docs/readme.go` — `UpdateReadmes`, `GenerateReadme`, `renderReadme`
- `internal/docs/input_docs.go` — `renderInputDocs`, `findDataStreamInputs`
- `internal/docs/exported_fields.go` — `renderExportedFields`
- `internal/requiredinputs/streamdefs.go` — `resolveStreamInputTypes` (writes to build root)
- `internal/requiredinputs/fields.go` — `bundleDataStreamFields` (writes to build root)
- `internal/builder/packages.go` — `BuildPackage` (pipeline ordering)
## Fix direction
Thread `buildPackageRoot` through `GenerateReadme` and `renderReadme`. Have the `inputDocs` and `fields` closures prefer `buildPackageRoot` when non-empty, falling back to `packageRoot` for the lint path (`AreReadmesUpToDate` has no build root available).
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.