Create new filter plugin: logstash-filter-excerpt
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 88
Description
There are situations where users process large events through Logstash and index it on to Elasticsearch. Typically you don't need the entire event indexed and searchable, but you'd still need to store the event for compliancy. It would be nice to have a filter that clones a big event, extracts an excerpt from it and indexes this to ES. Naively, an extraction can be the first 100 chars, or we can get fancier with it. Maybe even logstash-filter-summarization plugin based on [automatic summarization](https://en.wikipedia.org/wiki/Automatic_summarization). The original, large event can be routed to a long term storage like S3 or HDFS.
You can still achieve this today by tagging (based on size), cloning and using a ruby filter.
Contributor guide
Assessment
This issue has not been assessed yet.