Azure / Azure/azure-diagnostics-tools

Receiving "missing class name" errors when running Logstash with Azure blob input

Open
#176 0 comments 0 reactions 0 assignees View on GitHub
logstash-input-azureblobs
Dominant language
Ruby
Stars
101
Forks
93
PR merge metrics
No merged PRs in 30d

Description

When I run logstash with the Azure blob input configured, I get the following error:

2018-10-10T11:42:18,296][ERROR][logstash.inputs.logstashinputazureblob] Oh My, An error occurred. Error:missing class name (`javax.json.Json'): Trace: ["org/jruby/javasupport/JavaPackage.java:127:in `const_missing'", "C:/ELK/Logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-azureblob-0.9.13-java/lib/com/microsoft/json-parser.rb:22:in `initialize'", "C:/ELK/Logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-azureblob-0.9.13-java/lib/logstash/inputs/azureblob.rb:192:in `process'", "C:/ELK/Logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-azureblob-0.9.13-java/lib/logstash/inputs/azureblob.rb:151:in `run'", "C:/ELK/Logstash/logstash-core/lib/logstash/pipeline.rb:512:in `inputworker'", "C:/ELK/Logstash/logstash-core/lib/logstash/pipeline.rb:505:in `block in start_input'"] {:exception=>#}

Some more info in case it helps... I only got to this point after:

- Commenting out line 3 of the "json-parser.rb" file to resolve the error reported in #170
- Running logstash on a 64 bit Java installation as I was getting out of memory errors when running in 32 bit mode (even with the heap size adjusted as high as it could go)

What I'm trying to do is ingest NSG flow logs, similar to what is described [here](https://docs.microsoft.com/en-us/azure/network-watcher/network-watcher-visualize-nsg-flow-logs-open-source-tools).

My config:

input {
azureblob
{
storage_account_name => "xxxxxxxxx"
storage_access_key => "xxxxxxxxx"
container => "xxxxxxxxxxx"
codec => "json"
}
}

filter {
split { field => "[records]" }
split { field => "[records][properties][flows]"}
split { field => "[records][properties][flows][flows]"}
split { field => "[records][properties][flows][flows][flowTuples]"}


mutate {
split => { "[records][resourceId]" => "/"}
add_field => {"Subscription" => "%{[records][resourceId][2]}"
"ResourceGroup" => "%{[records][resourceId][4]}"
"NetworkSecurityGroup" => "%{[records][resourceId][8]}"}
convert => {"Subscription" => "string"}
convert => {"ResourceGroup" => "string"}
convert => {"NetworkSecurityGroup" => "string"}
split => { "[records][properties][flows][flows][flowTuples]" => ","}
add_field => {
"unixtimestamp" => "%{[records][properties][flows][flows][flowTuples][0]}"
"srcIp" => "%{[records][properties][flows][flows][flowTuples][1]}"
"destIp" => "%{[records][properties][flows][flows][flowTuples][2]}"
"srcPort" => "%{[records][properties][flows][flows][flowTuples][3]}"
"destPort" => "%{[records][properties][flows][flows][flowTuples][4]}"
"protocol" => "%{[records][properties][flows][flows][flowTuples][5]}"
"trafficflow" => "%{[records][properties][flows][flows][flowTuples][6]}"
"traffic" => "%{[records][properties][flows][flows][flowTuples][7]}"
}
convert => {"unixtimestamp" => "integer"}
convert => {"srcPort" => "integer"}
convert => {"destPort" => "integer"}
}


date{
match => ["unixtimestamp" , "UNIX"]
}
}


output {
file {
path => "c:\elk\output\nsglogs.txt"
}
}

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.