Logstash error ELK 5.4 - java.io.InvalidObjectException: Object type java.util.Hashtable is not allowed
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 19h 14m
- Merged PRs (30d)
- 63
Description
Hey,
i have two servers of applications (using the same version of application), and only on one of them i see this issue. Application is stored in Weblogic. They may differ slightly by `configuration ear` - but i did not find there anything that would have meaning in this matter.
[2017-06-26T11:16:29,552][DEBUG][logstash.inputs.log4j ] Accepted connection {:client=>"X", :server=>"0.0.0.0:4561"}
[2017-06-26T11:16:29,572][DEBUG][logstash.inputs.log4j ] Closing connection {:client=>"X", :exception=>java.io.InvalidObjectException: Object type java.util.Hashtable is not allowed.}
not sure what is the cause ?
`ELK 5.4` - elsewhere working fine
Configuration on both of them is the same.
What can be wrong in one of them ?
I tailed the log - and both seems to send the same log - but one is ok - the second one is not ok.
I run logstash by these commands:
bin/logstash -f centralLogstash.conf --debug --path.data usr/share/logstash/tmp
bin/logstash -f tcp_agent_4561.conf --debug --path.data usr/share/logstash/tmp2
`centralLogstash.conf`
input {
stomp{
password => "admin"
user => "admin"
host => "X"
destination => "stomp"
codec => "json"
}
}
output {
elasticsearch {
hosts => "X"
codec => "plain"
}
}
`
tcp_agent_4561.conf`
input {
log4j {
port => 4561
codec => multiline{
pattern => "^%{TIMESTAMP_ISO8601}"
negate => "true"
what => previous
}
}
}
filter {
if[priority] == "ERROR" {
mutate {
add_tag => [ "error" ]
}
grok {
match => ["stack_trace", "(?%{JAVAEXCEPTION})"]
}
}
mutate {
add_field => { "WeblogicName" => "X" "Appender" => "Y" }
remove_field => [ "path", "method", "file", "class", "thread" ]
}
}
output {
stomp {
password => "X"
user => "X"
host => "X"
destination => "stomp"
codec => "json"
}
}
Java configuration for both of application is the same:
package X.utils;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.log4j.net.SocketAppender;
import X.ApplicationProperties;
public class Log4jInitialization {
public static void init(ApplicationProperties properties){
String LOGSTASH_HOST = properties.getLOGSTASH_HOST();
String LOGSTASH_PORT_1 = properties.getLOGSTASH_PORT_1();
String LOGSTASH_PORT_2 = properties.getLOGSTASH_PORT_2();
String LOGSTASH_PORT_3 = properties.getLOGSTASH_PORT_3();
if (LOGSTASH_HOST == null || LOGSTASH_PORT_1 == null ||
LOGSTASH_PORT_2 == null ||LOGSTASH_PORT_3 == null){
return;
}
if (LOGSTASH_HOST.equals("") || LOGSTASH_PORT_1.equals("") ||
LOGSTASH_PORT_2.equals("") ||LOGSTASH_PORT_3.equals("")){
return;
}
int PORT_1 = Integer.parseInt(LOGSTASH_PORT_1);
int PORT_2 = Integer.parseInt(LOGSTASH_PORT_2);
int PORT_3 = Integer.parseInt(LOGSTASH_PORT_3);
SocketAppender Y = new SocketAppender();
Y.setRemoteHost(LOGSTASH_HOST);
Y.setPort(PORT_1);
Y.setReconnectionDelay(10000);
Y.setThreshold(Level.DEBUG);
Y.setName("Y");
Y.activateOptions();
Logger z = Logger.getLogger("Z");
z.addAppender(Y);
SocketAppender g = new SocketAppender();
g.setRemoteHost(LOGSTASH_HOST);
g.setPort(PORT_2);
g.setReconnectionDelay(10000);
g.setThreshold(Level.DEBUG);
g.setName("g");
g.activateOptions();
Logger f = Logger.getLogger("G");
f.addAppender(g);
SocketAppender l = new SocketAppender();
l.setRemoteHost(LOGSTASH_HOST);
l.setPort(PORT_3);
l.setReconnectionDelay(10000);
l.setThreshold(Level.DEBUG);
l.setName("l");
l.activateOptions();
Logger la = Logger.getLogger("L");
la.addAppender(l);
}
}
Both of them share the same application.properties file regarding these properties.
Both of them send pretty much the same message (except for time)
changed some stuff to `X` `Y` etc. just in case.
I'm not sure how to find reason why this is happening - any help narrowing down the issue will be great.
Contributor guide
Assessment
This issue has not been assessed yet.