elastic / elastic/logstash

Add a simulate endpoint for Logstash

Open
#7,832 6 comments 2 reactions 1 assignee Claimed by @jsvd View on GitHub
enhancement v6.1.0
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

Add a `simulate` endpoint as a convenience to test Logstash's filters in a pipeline. This is inspired from ingest's [simulate API](https://www.elastic.co/guide/en/elasticsearch/reference/master/simulate-pipeline-api.html) which is super useful. Today, if someone wants to test filters in dev/staging, they'd have to add `stdin` and `stdout` which means changing the existing pipeline config. This API can make it easy to test iterations to their config.

## Behavior

1. Using a named pipeline which has already been loaded from `pipeline.yml`.

**Note:** If the named pipeline has inputs/outputs, we ignore that and **don't** use them. This operation should have no side-effects to the original pipeline.

Request

```sh
POST localhost:9600/_node/pipeline//_simulate
{
"events": [
"183.60.215.50 - - [11/Sep/2014:22:00:00 +0000] \"GET /scripts/netcat-webserver HTTP/1.1\" 200 182 \"-\" \"Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html)"
]
}
```

Response is JSON formatted output of the Event(s) after running it through the filters.

```json
{
"events": [
{
"host": "node1",
"blah": "foo"
}
]
```

--------------------------------------
2. Using a pipeline definition provided via the body:

```sh
POST localhost:9600/_node/pipeline/_simulate
{
// pipeline definition as a string
"pipeline_config" : "filters { geoip { source => \"clientIP\" } }"
"events": [
"183.60.215.50 - - [11/Sep/2014:22:00:00 +0000] \"GET /scripts/netcat-webserver HTTP/1.1\" 200 182 \"-\" \"Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html)"
]
}
```

Response is JSON formatted output of the Event(s) after running it through the filters.

```json
{
"events": [
{
"foo": "bar",
"blah": "foo"
}
]
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.