`elastic-package create` supports composable package wizard

Summary

The composable build/merge path shipped in #3459 / #3474 / #3480 (umbrella #3277) but elastic-package create ... doesn't know composable exists. Two scaffolders need to learn the new shape:

1. elastic-package create package always emits the legacy policy-template input shape (type: logfile) and never a requires: block. There's no flag to opt into composable.
2. elastic-package create data-stream rejects otelcol (and logfile) outright, and even when fed an allowed --inputs value it dumps ~25 input vars + agent/stream/<input>.yml.hbs + elasticsearch/ingest_pipeline/default.yml into the stream — all of which come from the input package at build time for composable streams.

These two land together because the data-stream scaffolder needs to read the parent manifest's requires.input[] to validate --package, so the package-side change is a prerequisite.

Repro

cd /tmp && mkdir -p composable_repro/packages && cd composable_repro/packages

# Symptom A1: no composable shape from `create package`
elastic-package create package --name foo_composable --type integration
grep -A2 'inputs:' foo_composable/manifest.yml
# observe: type: logfile, no requires: block
cd foo_composable

# Symptom B1: otelcol rejected
elastic-package create data-stream --type logs --inputs otelcol
# error: invalid input type "otelcol"; allowed values: [aws-cloudwatch ...]

# Symptom B2: even allowed inputs dump stream cruft
elastic-package create data-stream --type logs --inputs filestream --name foo
ls foo/  # observe agent/stream/, elasticsearch/ingest_pipeline/, populated vars block

Proposed shape

elastic-package create package --composable

New boolean flag, default false. When set:

• Wizard prompts Required input packages? (y/N). If yes, loop prompting for <package_name>@<version> pairs.
• manifest.yml is rendered with:
  • A requires.input: block listing each captured pair.
  • policy_templates[].inputs[].package: <name> (matched 1:1 with the declared requires.input[] entries) instead of the legacy type: <input_type>.
• requires.content is not prompted (out of scope).
• Without --composable, current scaffolder output is byte-for-byte unchanged.

elastic-package create data-stream --package <input_pkg_name>

New flag, mutually exclusive with --inputs. When set:

• Skip the --inputs requirement (no allowlist check).
• Validate that <input_pkg_name> is in the parent manifest.yml's requires.input[]. Read ../manifest.yml relative to the new data-stream's parent dir. Error with a clear message if missing.
• Emit a minimal streams[0]: package: <input_pkg_name> plus title: and description:. Skip:
  • agent/stream/*.yml.hbs emission
  • elasticsearch/ingest_pipeline/*.yml emission
  • All copied vars from the input type (composable build merges them at build time).
• Author adds stream-specific vars (e.g. include: path overrides) manually post-scaffold.

Source pointers

elastic-package create package:

• Template: internal/packages/archetype/_static/package-manifest.yml.tmpl:33-43 (policy_templates / integration branch).
• Wizard/CLI: cmd/create_package.go. Version default 0.0.1 at :102 and :214 is fine.
• Either branch in-place on a new .Manifest.Composable boolean, or ship a parallel package-manifest-composable.yml.tmpl and pick by flag in internal/packages/archetype/package.go.

elastic-package create data-stream:

• Allowlist: internal/packages/packages.go:51 (AllowedLogsInputTypes) — otelcol/logfile absent.
• Validation site: cmd/create_data_stream.go:188-189.
• Wizard multi-select: cmd/create_data_stream.go:107.
• Var injection (the cruft): internal/packages/archetype/data_stream_inputs.go:48 (populateInputs).
• Templates to skip when --package is set: internal/packages/archetype/_static/dataStream-agent-stream.yml.tmpl, internal/packages/archetype/_static/dataStream-elasticsearch-ingest-pipeline.yml.tmpl.

Acceptance criteria

• elastic-package create package --composable --name foo --type integration produces a manifest.yml with at least one policy_templates[].inputs[].package: <name> (no legacy type:) and a requires.input: block whose entries match the policy_template inputs.
• Without --composable, elastic-package create package output is byte-for-byte unchanged.
• elastic-package create data-stream --type logs --package filelog_otel --name access succeeds when the parent's requires.input lists filelog_otel; fails with a clear error otherwise.
• The scaffolded composable data stream contains only manifest.yml (with streams[0].package:) — no agent/stream/, no elasticsearch/ingest_pipeline/, no copied vars.
• --package and --inputs are mutually exclusive; passing both errors at validation.
• Existing --inputs paths are byte-for-byte unchanged.
• elastic-package build -v on a package scaffolded with --composable + composable data streams succeeds with no manual edits (assuming declared input packages exist in the EPR).

Related

• #3000 (otelcol input-type selection — adjacent scope; decide if it merges into this flow).
• #3277 (composable build/merge umbrella).
• #3459, #3474, #3480 (composable build PRs already shipped).

Start with cmd/create_package.go and cmd/create_data_stream.go, then inspect the manifest and data-stream templates under internal/packages/archetype/_static plus data_stream_inputs.go. Trace the existing --inputs validation and rendering flow before adding the composable branches. Done means the listed acceptance cases pass, including unchanged legacy output, requires.input validation, mutually exclusive flags, minimal composable streams, and a successful verbose build.