aws / aws/amazon-documentdb-jdbc-driver

Logstash Pipeline : input jdbc - Must only be called before \"cook()\" | [BUG]

Open
#464 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
17
Forks
16
PR merge metrics
No merged PRs in 30d

Description

## Environment
- DocumentDB JDBC driver version: 1.4.2
- DocumentDB server version: MongoDB 4.0 Compatibility
- OS: Ubuntu 20.04
- BI Tool or client name: Logstash
- BI Tool or client version:
- Java version (if known): openjdk 11.0.17

---

## Problem Description
1. Steps to reproduce:

I am trying to run logstash pipeline which requires to get input (data from DocumentDB 'Mongo') and output to elasicSearch.

```
input {
jdbc{
jdbc_driver_library => ""
jdbc_driver_class => "software.amazon.documentdb.jdbc.DocumentDbMain"
jdbc_connection_string => "jdbc:documentdb://user:password@xxxxxxxxxxxxxxxxxxxxx.docdb.amazonaws.com/testDB?tlsAllowInvalidHostnames=true&tlsCAFile=rds-combined-ca-bundle.pem>
jdbc_user => ""
jdbc_password => ""
statement => "select * FROM `products` where `id` = 10"
}
}
filter {}
output{
stdout { codec => rubydebug { metadata=> true } }
elasticsearch {
action => "update"
doc_as_upsert => true
index => "logstash-products-index"
hosts => ["localhost:9200"]
user => ""
password => ""
}
}
```

3. Expected behaviour:

Get product from DocumentDB and add to ElasticSearch.

5. Actual behaviour:

When the pipeline is run, it fails and throws error.

7. Error message/stack trace:

[ERROR] 2022-12-19 14:52:25.217 [Ruby-0-Thread-17: :1] javapipeline - Worker loop initialization error {:pipeline_id=>"main", :error=>"Must only be called before \"cook()\"", :exception=>Java::JavaLang::IllegalStateException, :stacktrace=>"org.codehaus.janino.SimpleCompiler.assertUncooked(org/codehaus/janino/SimpleCompiler.java:526)\norg.codehaus.janino.SimpleCompiler.cook(org/codehaus/janino/SimpleCompiler.java:252)\norg.codehaus.janino.SimpleCompiler.compileToClassLoader(org/codehaus/janino/SimpleCompiler.java:517)\norg.codehaus.janino.SimpleCompiler.cook(org/codehaus/janino/SimpleCompiler.java:241)\norg.codehaus.janino.SimpleCompiler.cook(org/codehaus/janino/SimpleCompiler.java:219)\norg.codehaus.commons.compiler.Cookable.cook(org/codehaus/commons/compiler/Cookable.java:79)\norg.codehaus.commons.compiler.Cookable.cook(org/codehaus/commons/compiler/Cookable.java:74)\norg.logstash.config.ir.compiler.ComputeStepSyntaxElement.lambda$compile$0(org/logstash/config/ir/compiler/ComputeStepSyntaxElement.java:150)\njava.util.concurrent.ConcurrentHashMap.computeIfAbsent(java/util/concurrent/ConcurrentHashMap.java:1708)\norg.logstash.config.ir.compiler.ComputeStepSyntaxElement.compile(org/logstash/config/ir/compiler/ComputeStepSyntaxElement.java:140)\norg.logstash.config.ir.compiler.ComputeStepSyntaxElement.instantiate(org/logstash/config/ir/compiler/ComputeStepSyntaxElement.java:124)\norg.logstash.config.ir.compiler.DatasetCompiler.terminalOutputDataset(org/logstash/config/ir/compiler/DatasetCompiler.java:212)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.compileOutputs(org/logstash/config/ir/CompiledPipeline.java:414)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.(org/logstash/config/ir/CompiledPipeline.java:379)\norg.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.(org/logstash/config/ir/CompiledPipeline.java:337)\norg.logstash.config.ir.CompiledPipeline.buildExecution(org/logstash/config/ir/CompiledPipeline.java:156)\norg.logstash.execution.WorkerLoop.(org/logstash/execution/WorkerLoop.java:67)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(jdk/internal/reflect/NativeConstructorAccessorImpl.java:77)\njdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(jdk/internal/reflect/DelegatingConstructorAccessorImpl.java:45)\njava.lang.reflect.Constructor.newInstanceWithCaller(java/lang/reflect/Constructor.java:499)\njava.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:480)\norg.jruby.javasupport.JavaConstructor.newInstanceDirect(org/jruby/javasupport/JavaConstructor.java:237)\norg.jruby.RubyClass.new(org/jruby/RubyClass.java:911)\norg.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)\nusr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.init_worker_loop(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:580)\nusr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:286)\norg.jruby.RubyProc.call(org/jruby/RubyProc.java:309)\njava.lang.Thread.run(java/lang/Thread.java:833)", :thread=>"#"}

9. Any other details that can be helpful:
N/A

---

## Screenshots
N/A

---

## JDBC log

Contributor guide

Open the contributing guide

Research direction

No driver source files or tests are named; begin by reproducing the Logstash pipeline with the documented DocumentDB JDBC configuration and the listed Java and server versions. Trace the reported Janino and Logstash worker initialization path, then verify that the pipeline starts, retrieves the product, and sends it to Elasticsearch without the cook() error.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, java, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.