input-csvfile for Consideration
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 19h 14m
- Merged PRs (30d)
- 63
Description
I needed to process CSV files with first line schemas (that can vary from file-to-file) but see there's no clean way to do so with today's plugins. In some forum comments I see that improved CSV handling is on the "to do" list. I went ahead and coded-up a subclass of logstash-input-file that adds CSV parsing with a first-line-schema mode. The fundamental CSV parsing logic is largely borrowed from logstash-filter-csv, albeit extended with a first-line-schema mode.
I'd be interested in having it considered for submission as a standard plugin. Code/tests are at https://github.com/jweite/logstash-input-csvfile1, and particularly to get some other eyes on my code, as it's my first Logstash plugin.
PS, while I initially considered enhancing logstash-filter-csv I ultimately concluded that the only 100% reliable way to restart stream processing mid-file was to re-read the file's schema row, something that only the file input plugin can always do.
Contributor guide
Assessment
This issue has not been assessed yet.