elastic / elastic/elastic-package

README template functions {{ inputDocs }} and {{ fields }} produce incomplete output for composable (input-package) integrations

Open
#3,696 0 comments 0 reactions 1 assignee Claimed by @teresaromero View on GitHub
Dominant language
Go
Stars
72
Forks
141
Avg merge
19h 42m
Merged PRs (30d)
55

Description

## Summary

`{{ inputDocs }}` and `{{ fields }}` README template functions produce incomplete output for integrations that use composable (input-package) packages. `{{ inputDocs }}` always renders empty, and `{{ fields }}` silently omits any fields contributed by the required input package.

## Background

Composable integrations declare their inputs via `streams[].package: ` in data stream manifests instead of the conventional `streams[].input`. During `elastic-package build`, `RequiredInputsResolver.Bundle` rewrites those references to `streams[].input` and copies field definitions from the input package into the data stream's `fields/` directory — but both operations happen exclusively in the **build root** (`build/packages///`). The source tree is never modified.

`UpdateReadmes` is called after `Bundle` and correctly receives `buildPackageRoot`, but it passes only `packageRoot` (source) to `GenerateReadme` → `renderReadme`, where all template closures read their data. Because the composable-specific data never reaches the source tree, both affected functions return incomplete results.

## Affected template functions

| Template function | What it reads | What it misses |
|---|---|---|
| `{{ inputDocs }}` | `data_stream/*/manifest.yml` from source for `streams[].input` | `streams[].input` — absent in source; only written to build root by `resolveStreamInputTypes` |
| `{{ fields }}` | `data_stream/*/fields/*.yml` from source | `-fields.yml` — written only to build root by `bundleDataStreamFields` |

Unaffected: `{{ ilm }}`, `{{ transform }}`, `{{ event }}`, `{{ alertRuleTemplates }}`, `{{ sloTemplates }}`.

## Observed behaviour

- `{{ inputDocs }}` renders as an empty string for any package that uses `streams[].package`.
- `{{ fields }}` omits fields that originate from the required input package (defined in the input package's `fields/` directory and not overridden in the composable integration).

Discovered while working on https://github.com/elastic/integrations/pull/19719 (`elastic_package_registry`, which uses `streams[].package: prometheus_input`).

## Expected behaviour

Both `{{ inputDocs }}` and `{{ fields }}` should produce the same output that a conventional (non-composable) integration with equivalent resolved manifests and fields would produce.

## Root cause

`internal/docs/readme.go` — `renderReadme` registers all template closures over `packageRoot` (source path). `buildPackageRoot` is available at the `UpdateReadmes` call site (`internal/builder/packages.go:259`) but is not threaded into `GenerateReadme` or `renderReadme`.

Relevant code:
- `internal/docs/readme.go` — `UpdateReadmes`, `GenerateReadme`, `renderReadme`
- `internal/docs/input_docs.go` — `renderInputDocs`, `findDataStreamInputs`
- `internal/docs/exported_fields.go` — `renderExportedFields`
- `internal/requiredinputs/streamdefs.go` — `resolveStreamInputTypes` (writes to build root)
- `internal/requiredinputs/fields.go` — `bundleDataStreamFields` (writes to build root)
- `internal/builder/packages.go` — `BuildPackage` (pipeline ordering)

## Fix direction

Thread `buildPackageRoot` through `GenerateReadme` and `renderReadme`. Have the `inputDocs` and `fields` closures prefer `buildPackageRoot` when non-empty, falling back to `packageRoot` for the lint path (`AreReadmesUpToDate` has no build root available).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.