elastic / elastic/elastic-serverless-forwarder

Add `pipeline:` field to Elastic Serverless Forwarder specification

Open
#464 9 comments 3 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
38
Forks
50
PR merge metrics
No merged PRs in 30d

Description

## Use Case

As the administrator of our Elastic Cloud environment, I need the capability for my developers to specify a custom Elasticsearch Ingest pipeline for their logs in the ESF `config.yml`, similar to Elastic Agent or Functionbeat.

## Describe the Enhancement

Currently, in order to use Elastic Serverless Forwarder with a custom ingest pipeline, it is necessary to deploy a custom log template (e.g., `logs-foo.log@custom`) This is not required with Elastic Agent or Functionbeat, which allow this to be done with the `pipeline:` specification. Please add this capability to the Elastic Serverless Forwarder.

An example of what this might look like is:

```yaml
---
inputs:
- type: 'cloudwatch-logs'
id: ''
json_content_type: 'ndjson'
pipeline: 'my-pipeline' # <-- custom Elasticsearch ingest pipeline
outputs:
- type: 'elasticsearch'
args:
elasticsearch_url: https://elastic-cloud:443
api_key: 'my-api-key'
es_datastream_name: 'logs-myapp.log-default'
batch_max_actions: 500
batch_max_bytes: 10485760
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.