elastic / elastic/logstash

Logstash error ELK 5.4 - java.io.InvalidObjectException: Object type java.util.Hashtable is not allowed

Open
#7,535 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

Hey,
i have two servers of applications (using the same version of application), and only on one of them i see this issue. Application is stored in Weblogic. They may differ slightly by `configuration ear` - but i did not find there anything that would have meaning in this matter.

[2017-06-26T11:16:29,552][DEBUG][logstash.inputs.log4j ] Accepted connection {:client=>"X", :server=>"0.0.0.0:4561"}
[2017-06-26T11:16:29,572][DEBUG][logstash.inputs.log4j ] Closing connection {:client=>"X", :exception=>java.io.InvalidObjectException: Object type java.util.Hashtable is not allowed.}

not sure what is the cause ?
`ELK 5.4` - elsewhere working fine
Configuration on both of them is the same.

What can be wrong in one of them ?
I tailed the log - and both seems to send the same log - but one is ok - the second one is not ok.

I run logstash by these commands:

bin/logstash -f centralLogstash.conf --debug --path.data usr/share/logstash/tmp

bin/logstash -f tcp_agent_4561.conf --debug --path.data usr/share/logstash/tmp2

`centralLogstash.conf`

input {
stomp{
password => "admin"
user => "admin"
host => "X"
destination => "stomp"
codec => "json"
}
}

output {
elasticsearch {
hosts => "X"
codec => "plain"
}
}

`
tcp_agent_4561.conf`

input {
log4j {
port => 4561
codec => multiline{
pattern => "^%{TIMESTAMP_ISO8601}"
negate => "true"
what => previous
}
}
}

filter {
if[priority] == "ERROR" {
mutate {
add_tag => [ "error" ]
}
grok {
match => ["stack_trace", "(?%{JAVAEXCEPTION})"]
}
}

mutate {
add_field => { "WeblogicName" => "X" "Appender" => "Y" }
remove_field => [ "path", "method", "file", "class", "thread" ]
}
}

output {
stomp {
password => "X"
user => "X"
host => "X"
destination => "stomp"
codec => "json"
}
}

Java configuration for both of application is the same:

package X.utils;

import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.log4j.net.SocketAppender;

import X.ApplicationProperties;

public class Log4jInitialization {

public static void init(ApplicationProperties properties){

String LOGSTASH_HOST = properties.getLOGSTASH_HOST();
String LOGSTASH_PORT_1 = properties.getLOGSTASH_PORT_1();
String LOGSTASH_PORT_2 = properties.getLOGSTASH_PORT_2();
String LOGSTASH_PORT_3 = properties.getLOGSTASH_PORT_3();

if (LOGSTASH_HOST == null || LOGSTASH_PORT_1 == null ||
LOGSTASH_PORT_2 == null ||LOGSTASH_PORT_3 == null){
return;
}

if (LOGSTASH_HOST.equals("") || LOGSTASH_PORT_1.equals("") ||
LOGSTASH_PORT_2.equals("") ||LOGSTASH_PORT_3.equals("")){
return;
}

int PORT_1 = Integer.parseInt(LOGSTASH_PORT_1);
int PORT_2 = Integer.parseInt(LOGSTASH_PORT_2);
int PORT_3 = Integer.parseInt(LOGSTASH_PORT_3);

SocketAppender Y = new SocketAppender();
Y.setRemoteHost(LOGSTASH_HOST);
Y.setPort(PORT_1);
Y.setReconnectionDelay(10000);
Y.setThreshold(Level.DEBUG);
Y.setName("Y");
Y.activateOptions();

Logger z = Logger.getLogger("Z");
z.addAppender(Y);

SocketAppender g = new SocketAppender();
g.setRemoteHost(LOGSTASH_HOST);
g.setPort(PORT_2);
g.setReconnectionDelay(10000);
g.setThreshold(Level.DEBUG);
g.setName("g");
g.activateOptions();

Logger f = Logger.getLogger("G");
f.addAppender(g);

SocketAppender l = new SocketAppender();
l.setRemoteHost(LOGSTASH_HOST);
l.setPort(PORT_3);
l.setReconnectionDelay(10000);
l.setThreshold(Level.DEBUG);
l.setName("l");
l.activateOptions();

Logger la = Logger.getLogger("L");
la.addAppender(l);

}
}

Both of them share the same application.properties file regarding these properties.

Both of them send pretty much the same message (except for time)

changed some stuff to `X` `Y` etc. just in case.

I'm not sure how to find reason why this is happening - any help narrowing down the issue will be great.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.