opensearch-project / opensearch-project/observability-stack

error parsing for otel-v1-apm-span-000001 using samples configs provided for data prepper.

Open
#197 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
JavaScript
Stars
41
Forks
39
Avg merge
2d 11h
Merged PRs (30d)
10

Description

I installed opensearch and dashboard 3.6 and followed config samples from observiblity-stack github. I have OTEL collector , data prepper and prometheus started fine.

Upon starting data prepper i see this

2026-04-14T23:39:30,387 [traces-raw-pipeline-sink-worker-8-thread-1] INFO org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager - Found version 1 for existing index template otel-v1-apm-span-index-template
2026-04-14T23:39:30,389 [traces-raw-pipeline-sink-worker-8-thread-1] INFO org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager - Index template otel-v1-apm-span-index-template should not be updated, current version 1 >= existing version 1

But when i test my app i see following error in data prepper log.
2026-04-14T23:33:16,189 [traces-raw-pipeline-sink-worker-8-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Document failed to write to OpenSearch with error code 400. Configure a DLQ to save failed documents. Error: failed to parse caused by class org.opensearch.index.mapper.KeywordFieldMapper cannot be cast to class org.opensearch.index.mapper.ObjectMapper (org.opensearch.index.mapper.KeywordFieldMapper and org.opensearch.index.mapper.ObjectMapper are in unnamed module of loader 'app')

then i enabled dlq and after restart of data prepper i see following error.

2026-04-14T23:40:33,839 [traces-raw-pipeline-sink-worker-8-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.BulkRetryStrategy - index = otel-v1-apm-span-000001, operation = Index, status = 400, error = failed to parse

I already tried deleting auto created index , index alias but no luck.

any idea on what is wrong here ?

Data prepper version 2.15

doc link https://docs.opensearch.org/latest/observing-your-data/apm/index/

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the sample configurations linked from the observability-stack repository and the APM documentation, then compare the otel-v1-apm-span index template and mapping with the failed document in the Data Prepper logs. Reproduce the trace ingestion failure and verify that the resulting span is accepted by OpenSearch without the 400 parse error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, prometheus
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.