[RFC] Support `collect` command in PPL

Problem Statement

PPL currently has no way to persist query results back into an index. This limits workflows such as:

• Precomputing and storing expensive aggregations for reuse
• Accelerating dashboards or reports by materializing results
• Archiving or migrating subsets of data across indexes

The question is whether PPL should introduce a collect command (or something equivalent) to support these scenarios.

example syntax:

source=logs | stats count by type | collect index=summary

Current State

• PPL is designed as a read-only query language.
• Users can export results externally and re-ingest them into an index, but this adds friction and operational overhead.
• There is no in-query mechanism to take the output of a pipeline and persist it.

Open Questions / Discussion Points

• Do we need a collect command in PPL to directly write results into an index?

• What are the most common use cases for this command?

• Should this be limited to a required index= parameter only, or should we plan for extensibility later?

• Security concerns:

  • Should all users be allowed to write to indexes through queries, or should this be restricted by role/permissions?
  • How do we prevent accidental or malicious overwrites of data in critical indexes?
  • Do we need safeguards to restrict which indexes can be written to?

• Are there risks of performance degradation if large query outputs are collected repeatedly?

• How critical is this for users compared to relying on ingestion pipelines or external tools?

Long-Term Goals

If adopted, the goal would be:

• Provide a straightforward way to persist query results into an index.
• Ensure the feature is secure and respects existing access controls.

No implementation files or tests are identified in the issue. Start by reviewing the existing PPL command and index-write architecture, then clarify whether a collect command is wanted and how permissions, overwrite protection, and large outputs should behave. Done means an agreed design and an implementation scope with acceptance criteria.