elastic / elastic/logstash

Long lines in files crash logstash

Open
#4,009 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

We have a problem where some of our applications log very long lines into files (> 32k characters). This (I believe) is crashing logstash with:

SocketError: send: name or service not known
send at org/jruby/ext/socket/RubyUDPSocket.java:318
register at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-udp-1.0.0/lib/logstash/outputs/udp.rb:24
call at org/jruby/RubyProc.java:271
encode at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-json-1.0.1/lib/logstash/codecs/json.rb:57
receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-udp-1.0.0/lib/logstash/outputs/udp.rb:34
handle at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/outputs/base.rb:88
output_func at (eval):209
outputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:244
start_outputs at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:166

I saw this thread relating to logstash-forwarder:

https://github.com/elastic/logstash-forwarder/issues/33

But I'm not using this. I also found this StackOverflow post which indicates the maximum line length:

http://stackoverflow.com/questions/31299262/maximum-line-length-supported-by-logstash

I'm not sure what's correct, whether to truncate the line, or increase the maximum length, but it would be nice if it didn't crash. We're also modifying our application to remove the log line, which should never have been there in the first place.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.