aws / aws/amazon-documentdb-jdbc-driver
Logstash Pipeline : input jdbc - Must only be called before \"cook()\" | [BUG]
- Dominant language
- Java
- Stars
- 17
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
## Environment
- DocumentDB JDBC driver version: 1.4.2
- DocumentDB server version: MongoDB 4.0 Compatibility
- OS: Ubuntu 20.04
- BI Tool or client name: Logstash
- BI Tool or client version:
- Java version (if known): openjdk 11.0.17
---
## Problem Description
1. Steps to reproduce:
I am trying to run logstash pipeline which requires to get input (data from DocumentDB 'Mongo') and output to elasicSearch.
```
input {
jdbc{
jdbc_driver_library => ""
jdbc_driver_class => "software.amazon.documentdb.jdbc.DocumentDbMain"
jdbc_connection_string => "jdbc:documentdb://user:password@xxxxxxxxxxxxxxxxxxxxx.docdb.amazonaws.com/testDB?tlsAllowInvalidHostnames=true&tlsCAFile=rds-combined-ca-bundle.pem>
jdbc_user => ""
jdbc_password => ""
statement => "select * FROM `products` where `id` = 10"
}
}
filter {}
output{
stdout { codec => rubydebug { metadata=> true } }
elasticsearch {
action => "update"
doc_as_upsert => true
index => "logstash-products-index"
hosts => ["localhost:9200"]
user => ""
password => ""
}
}
```
3. Expected behaviour:
Get product from DocumentDB and add to ElasticSearch.
5. Actual behaviour:
When the pipeline is run, it fails and throws error.
7. Error message/stack trace:
[ERROR] 2022-12-19 14:52:25.217 [Ruby-0-Thread-17: :1] javapipeline - Worker loop initialization error {:pipeline_id=>"main", :error=>"Must only be called before \"cook()\"", :exception=>Java::JavaLang::IllegalStateException, :stacktrace=>"org.codehaus.janino.SimpleCompiler.assertUncooked(org/codehaus/janino/SimpleCompiler.java:526)\norg.codehaus.janino.SimpleCompiler.cook(org/codehaus/janino/SimpleCompiler.java:252)\norg.codehaus.janino.SimpleCompiler.compileToClassLoader(org/codehaus/janino/SimpleCompiler.java:517)\norg.codehaus.janino.SimpleCompiler.cook(org/codehaus/janino/SimpleCompiler.java:241)\norg.codehaus.janino.SimpleCompiler.cook(org/codehaus/janino/SimpleCompiler.java:219)\norg.codehaus.commons.compiler.Cookable.cook(org/codehaus/commons/compiler/Cookable.java:79)\norg.codehaus.commons.compiler.Cookable.cook(org/codehaus/commons/compiler/Cookable.java:74)\norg.logstash.config.ir.compiler.ComputeStepSyntaxElement.lambda$compile$0(org/logstash/config/ir/compiler/ComputeStepSyntaxElement.java:150)\njava.util.concurrent.ConcurrentHashMap.computeIfAbsent(java/util/concurrent/ConcurrentHashMap.java:1708)\norg.logstash.config.ir.compiler.ComputeStepSyntaxElement.compile(org/logstash/config/ir/compiler/ComputeStepSyntaxElement.java:140)\norg.logstash.config.ir.compiler.ComputeStepSyntaxElement.instantiate(org/logstash/config/ir/compiler/ComputeStepSyntaxElement.java:124)\norg.logstash.config.ir.compiler.DatasetCompiler.terminalOutputDataset(org/logstash/config/ir/compiler/DatasetCompiler.java:212)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.compileOutputs(org/logstash/config/ir/CompiledPipeline.java:414)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.(org/logstash/config/ir/CompiledPipeline.java:379)\norg.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.(org/logstash/config/ir/CompiledPipeline.java:337)\norg.logstash.config.ir.CompiledPipeline.buildExecution(org/logstash/config/ir/CompiledPipeline.java:156)\norg.logstash.execution.WorkerLoop.(org/logstash/execution/WorkerLoop.java:67)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(jdk/internal/reflect/NativeConstructorAccessorImpl.java:77)\njdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(jdk/internal/reflect/DelegatingConstructorAccessorImpl.java:45)\njava.lang.reflect.Constructor.newInstanceWithCaller(java/lang/reflect/Constructor.java:499)\njava.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:480)\norg.jruby.javasupport.JavaConstructor.newInstanceDirect(org/jruby/javasupport/JavaConstructor.java:237)\norg.jruby.RubyClass.new(org/jruby/RubyClass.java:911)\norg.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)\nusr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.init_worker_loop(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:580)\nusr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:286)\norg.jruby.RubyProc.call(org/jruby/RubyProc.java:309)\njava.lang.Thread.run(java/lang/Thread.java:833)", :thread=>"#"}
9. Any other details that can be helpful:
N/A
---
## Screenshots
N/A
---
## JDBC log
Contributor guide
Research direction
No driver source files or tests are named; begin by reproducing the Logstash pipeline with the documented DocumentDB JDBC configuration and the listed Java and server versions. Trace the reported Janino and Logstash worker initialization path, then verify that the pipeline starts, retrieves the product, and sends it to Elasticsearch without the cook() error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, java, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100