elastic / elastic/logstash

The latest multiline message does not go through FILTER after SIGINT received

Open
#4,690 5 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

Hello.

I have **Logstash 2.2.1**.

Also use a **lumberjack** protocol as input with **multiline codec**:

```
input {
lumberjack {
port => 5000
ssl_certificate => "....crt"
ssl_key => "....key"

codec => multiline {
charset => "CP1251"
patterns_dir => "/data/logstash/patterns"
pattern => "^\[%{WEBSPHERE_SYSTEMOUT_EVENTLOG}"
negate => true
what => "previous"
}
}
}
```

Some filters. Here is the mutate part

```
mutate {
add_field => {
"[@metadata][index]" => "%{index}"
"[@metadata][type]" => "%{type}"
}
}

```

And **stdout** as an output

```
output {
stdout { codec => rubydebug { metadata => true } }
}
```

It works fine, but when I send SIGINT to Logstash it seems that Logstash omits **filter** part for the latest and goes right to the **output** and there's no filter processing for %{index} and %{type} variables.

```
^C{
"@timestamp" => "2016-02-15T07:16:28.500Z",
"message" => "[15.02.16 10:16:28:500 GMT+03:00] 000000d6 ServletWrappe E com.ibm.ws.webcontainer.servlet.ServletWrapper service SRVE0068E",
"@version" => "1",
"@metadata" => {
"timestamp" => "15.02.16 10:16:28:500",
"index" => "%{index}",
"type" => "%{type}"
}
}
```

Contributor guide

Open the contributing guide

Research direction

Start with the multiline codec and the SIGINT shutdown path, then trace how the latest event moves through the configured filters and stdout rubydebug output. Reproduce the provided lumberjack configuration and verify that the final multiline message receives filter processing before it is emitted.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.