elastic / elastic/logstash

Add pipeline id and plugin id to logs

Open
#15,550 1 comment 0 reactions 0 assignees View on GitHub
enhancement status:needs-triage
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

When tracking down errors like:

```
[2023-11-08T14:10:18,726][ERROR][logstash.filters.ruby ] Ruby exception occurred: undefined method `[]' for nil:NilClass {:class=>"NoMethodError", :backtrace=>["(ruby filter code):10:in `block in filter_method'", [...]
```

It can be hard to track down the exact source, at least when the instance has many pipelines each with multiple ruby filter plugin definitions.

It would be very helpful if logs included pipeline id and plugin id (if available). E.g. an idea would be to have the above error message look somewhat like this instead:

```
[2023-11-08T14:10:18,726][ERROR][logstash.filters.ruby][my_pipeline_id][my_plugin_id] Ruby exception occurred: undefined method `[]' for nil:NilClass [...]
```

This would make it immensely easier to find the exact spot that triggered the error.

This is somewhat related to older issues:

- https://github.com/elastic/logstash/issues/4621
- https://github.com/elastic/logstash/issues/4277
- https://github.com/logstash-plugins/logstash-filter-ruby/issues/14

These old issues do wish for being able to find the source of errors more easily, and since these issues were last touched, it appears that the filter in question has been added to the logs (ie. now includes `logstash.filters.ruby` for the ruby filter), but I think they all miss this mention of pipeline and plugin ids specifically.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.