From Discuss: No events in DLQ
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 88
Description
from https://discuss.elastic.co/t/no-events-in-dlq/94309
User is trying to ingest data using the elasticsearch output plugin, and is seeing the following error from Elasticsearch:
```
MapperParsingException: object mapping for [mapping_request.payload] tried to parse field [payload] as object, but found a concrete value
```
but those entries are not being written to the DLQ as expected.
Some messages _have_ been written to the DLQ, but the user is seeing the following errors
```
[2017-07-25T16:41:19,389][ERROR][logstash.pipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Plugin: "/data/logstashData/dead_letter_queue", sincedb_path=>"/data/logstashDLQData/dlq_offset", commit_offsets=>true, codec=>"json_73337b45-c626-4cae-aab9-b2f4d8775a1e", enable_metric=>true, charset=>"UTF-8">, id=>"52ff26acd4e4d46066179db0a3cdf329cbe3c13c-1", enable_metric=>true, pipeline_id=>"main">
```
Contributor guide
Research direction
Start with the linked Discuss report and the dead_letter_queue plugin error, using the Elasticsearch mapping exception as the reproduction symptom. Trace why those failed events are not written to the DLQ and why the plugin restarts; done means the reported failures are handled as expected and the existing DLQ behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100