elastic / elastic/logstash

Lack of CR/LF in file stops processing

Open
#2,505 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

I've been playing with a few things around some open data sets, specifically [this one](http://data.gov.au/dataset/disaster-events-with-category-impact-and-location/resource/ad5c6594-571e-4874-994c-a9f964d789df). What I am finding is that if I query this API and get a dataset and save to a file, there is no CR/LF. Then when I try to then push this through Logstash I get nothing!
# Curl to save to file;

``` bash
curl -s http://data.gov.au/api/action/datastore_search?resource_id=ad5c6594-571e-4874-994c-a9f964d789df -o raw.out
```
# Basic Logstash config;

```
input { stdin {} }
output {stdout {} }
```
# Test

``` bash
$ cat ../raw.out |bin/logstash agent -f conf/basic.conf
$
```

I've tried to work around this via a few methods using the ruby filter;

```
filter {
ruby {
#code => "event['message'] << '\n\r'"
# Or
#code => "event['message'] = event['message'] + '\n\r'"
}
}
```

But still get no output.

I am certain this is a CR/LF issue, as if I vim the raw.out I see `[noeol]`, and then if I add a LF and push the file through the basic config I get the whole file output as the message.

This is using 1.5b1 on OS X.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.