elastic / elastic/logstash

Invalid update request to Elasticsearch when document field is named "data"

Open
#4,796 5 comments 0 reactions 1 assignee Claimed by @andrewvc View on GitHub
bug
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
19h 14m
Merged PRs (30d)
63

Description

When using logstash to update existing document, I want to add a field named `data`, but that results in an invalid bulk request to Elasticsearch. Is field name `data` reserved? The error from Elasticsearch is

```
IllegalArgumentException[Malformed action/metadata line [3], expected START_OBJECT or END_OBJECT but found [null]]
```

logstash outputs (formatted for readability):

```
Attempted to send a bulk request to Elasticsearch configured at '["http://localhost:9200/"]', but an error occurred and it failed! Are you sure you can reach elasticsearch from this machine using the configuration provided?
{
:client_config=>{
:hosts=>["http://localhost:9200/"],
:ssl=>nil,
:transport_options=>{:socket_timeout=>0, :request_timeout=>0, :proxy=>nil, :ssl=>{}},
:transport_class=>Elasticsearch::Transport::Transport::HTTP::Manticore,
:logger=>nil,
:tracer=>nil,
:reload_connections=>false,
:retry_on_failure=>false,
:reload_on_failure=>false,
:randomize_hosts=>false
},
:error_message=>"[500] {\"error\":\"IllegalArgumentException[Malformed action/metadata line [3], expected START_OBJECT or END_OBJECT but found [null]]\",\"status\":500}",
:error_class=>"Elasticsearch::Transport::Transport::Errors::InternalServerError",
:backtrace=>[
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:146:in `__raise_transport_error'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:256:in `perform_request'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/http/manticore.rb:54:in `perform_request'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/client.rb:125:in `perform_request'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.15/lib/elasticsearch/api/actions/bulk.rb:87:in `bulk'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:53:in `non_threadsafe_bulk'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:38:in `bulk'",
"org/jruby/ext/thread/Mutex.java:149:in `synchronize'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:38:in `bulk'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.1-java/lib/logstash/outputs/elasticsearch/common.rb:162:in `safe_bulk'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.1-java/lib/logstash/outputs/elasticsearch/common.rb:101:in `submit'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.1-java/lib/logstash/outputs/elasticsearch/common.rb:86:in `retrying_submit'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.1-java/lib/logstash/outputs/elasticsearch/common.rb:29:in `multi_receive'",
"org/jruby/RubyArray.java:1653:in `each_slice'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.1-java/lib/logstash/outputs/elasticsearch/common.rb:28:in `multi_receive'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/output_delegator.rb:119:in `worker_multi_receive'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/output_delegator.rb:65:in `multi_receive'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/pipeline.rb:290:in `output_batch'",
"org/jruby/RubyHash.java:1342:in `each'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/pipeline.rb:290:in `output_batch'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/pipeline.rb:221:in `worker_loop'",
"/Users/gg/Applications/logstash-2.2.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/pipeline.rb:190:in `start_workers'"
],
```

The update request looks like this:

```
bin/logstash -e 'input { stdin {codec => "json_lines"} } output { elasticsearch {hosts => ["localhost:9200"] action => "update" index => "logtest" document_type => "entry" document_id => "%{id}"} }' <

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.