Simplify Pipeline Creation

Is your feature request related to a problem? Please describe.

Creating a trace analytics pipeline requires a bit of configuration work. It should be possible to put together a trace analytics pipeline more easily and with less setup.

Right now, pipeline authors copy a 100-line code snippet from the Trace Analytics Setup documentation. Then they modify certain parts to fit their needs.

Describe the solution you'd like

The goal of this issue is to create a good solution for making pipeline configuration simpler.

Here is a possible configuration which could work:

trace_analytics_pipeline:
  type: pipeline_template
  source_configuration:
    ssl: true
    ssl_key_file: /my/path/file.key
    ssl_certificate_file: /my/path/file.cert
  sinks_configuration:
    hosts: [ "https://myhost:9200" ]
    username: "admin"
    password: "secret"
    cert: /my/path/to/cert

I'm proposing one solution, but we may be able to find another one. The solution I'm proposing first is what I call a "pipeline template". The idea is that we have a predefined pipeline. Then the values for it can map from a configuration which the "pipeline template" defines and then pipeline authors configure.

Importantly, the ability to compose the pipeline in a non-standard way is still there because we keep all the existing pipeline components (otel_trace_source, opensearch sink, etc.). This is just a new way to configure it.

Additional context

I created this issue as a placeholder and as a place to discuss possible solutions. There might be a better solution that we can arrive at.

Start with docs/trace_analytics.md and review the existing otel_trace_source and OpenSearch sink configuration described there. Compare the proposed pipeline_template configuration with the existing composition approach. This placeholder needs an agreed solution before implementation can be considered done.