opensearch-project/data-prepper

View on GitHub

OpenSearch Data Prepper is a component of the OpenSearch project that accepts, filters, transforms, enriches, and routes data at scale.

Stars: 374
Forks: 354
Open beginner issues: 2
Indexed issues: 636
Avg merge: 3d 18h
Merged PRs (30d): 8

Dominant language: Java
License: Apache-2.0
Last GitHub push: Sep 17, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: good first issue help wanted

Beginner friendly Everything

Open All

636 open issues indexed

otel_trace_source drops the whole OTLP request when a span has duplicate attribute keys Open

untriaged

Difficulty 3/5 1-2 days Newbie friendliness 72/100

opensearch-project/data-prepper#7151 · 1 comment · Opened Sep 19, 2026
[BUG]DynamoDB source: Stream shards never consumed after export completion when export takes >24h Open

bug untriaged

Difficulty 4/5 3-5 days Newbie friendliness 45/100

opensearch-project/data-prepper#7150 · Opened Sep 18, 2026
Support parquet / composite dataformat OTel indices in the OpenSearch sink Open

untriaged

Difficulty 4/5 3-5 days Newbie friendliness 52/100

opensearch-project/data-prepper#7142 · 1 comment · Opened Sep 15, 2026
[BUG] KafkaSink.doOutput can return while holding reentrantLock, and calls prepareTopicAndSchema() on every batch Open

bug untriaged

Difficulty 4/5 3-5 days Newbie friendliness 64/100

opensearch-project/data-prepper#7133 · Opened Sep 10, 2026
[BUG] KafkaCustomProducer.bufferedEventHandles is an unsynchronised LinkedList, causing ConcurrentModificationException and unreleased event handles Open

bug untriaged

Difficulty 4/5 3-5 days Newbie friendliness 58/100

opensearch-project/data-prepper#7132 · Opened Sep 10, 2026
[BUG] Kafka sink creates a new Kafka producer on every `doOutput` and never closes it, exhausting the thread limit* Open

bug untriaged

Difficulty 4/5 3-5 days Newbie friendliness 55/100

opensearch-project/data-prepper#7131 · Opened Sep 10, 2026
[BUG] Kafka sink throws NullPointerException on every shutdown when thread_wait_time is not set Open

bug untriaged

Difficulty 2/5 1-3 hours Newbie friendliness 88/100

opensearch-project/data-prepper#7130 · Opened Sep 10, 2026
Add merge_arrays processor and enhance convert_entry_type for nested keys in iterate_on Open

untriaged

Difficulty 5/5 Over a week Newbie friendliness 45/100

opensearch-project/data-prepper#7126 · Opened Sep 7, 2026
[BUG] RDS source (MySQL): stale in-memory tableId→metadata mapping causes rows from a non-synced table to be processed with a synced table's schema after a server restart Open

bug untriaged

Difficulty 4/5 3-5 days Newbie friendliness 58/100

opensearch-project/data-prepper#7125 · 2 comments · Opened Sep 3, 2026
Make DynamoDB stream shard acquisition backoff ceiling configurable Open

untriaged

Difficulty 3/5 1-2 days Newbie friendliness 70/100

opensearch-project/data-prepper#7124 · Opened Sep 3, 2026
Support Protobuf with Confluent Schema Registry in the Kafka source Open

untriaged

Difficulty 4/5 3-5 days Newbie friendliness 68/100

opensearch-project/data-prepper#7121 · Opened Sep 2, 2026
CVE-2026-75595 | Netty SSL vulnerability Open

untriaged

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

opensearch-project/data-prepper#7112 · Opened Aug 27, 2026
[BUG] Race Condition: NullPointerException in BulkIngester.output() When Sink Initialization Is Delayed Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 52/100

opensearch-project/data-prepper#7099 · Opened Aug 17, 2026
[Enhancement] Add overwrite_if_key_exists option to flatten processor for per-field merge behavior Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 68/100

opensearch-project/data-prepper#7093 · 2 comments · Opened Aug 13, 2026
[Enhancement] Add merge_map processor for merging maps/objects within an event Open

enhancement question

Difficulty 4/5 3-5 days Newbie friendliness 55/100

opensearch-project/data-prepper#7092 · 2 comments · Opened Aug 13, 2026
[BUG] RDS PostgreSQL source: CREATE PUBLICATION uses unquoted identifiers, failing on hyphenated or mixed-case database/schema/table names Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 42/100

opensearch-project/data-prepper#7077 · 1 comment · Opened Aug 6, 2026
[RFC] Vertex AI support in the ml_inference processor Open

enhancement follow up

Difficulty 5/5 Over a week Newbie friendliness 30/100

opensearch-project/data-prepper#7068 · Opened Aug 2, 2026
[RFC] Google Cloud Storage (GCS) Source and Sink Plugins Open

enhancement follow up

Difficulty 5/5 Over a week Newbie friendliness 35/100

opensearch-project/data-prepper#7067 · 3 comments · Opened Aug 2, 2026
[RFC] MQTT Source Plugin Open

enhancement follow up

Difficulty 5/5 Over a week Newbie friendliness 25/100

opensearch-project/data-prepper#7043 · Opened Jul 31, 2026
Use ShardFilter to reduce calls on DescribeStream Open

enhancement

opensearch-project/data-prepper#7041 · 1 assignee · Opened Jul 29, 2026
[BUG] Do not send 409 document already exists errors to DLQ Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 64/100

opensearch-project/data-prepper#7037 · Opened Jul 28, 2026
[BUG] Cannot include hidden indices Open

enhancement follow up question

Difficulty 3/5 1-2 days Newbie friendliness 55/100

opensearch-project/data-prepper#7026 · 3 comments · Opened Jul 24, 2026
Permissions issues with AOS and composable index template Open

question

Difficulty 4/5 3-5 days Newbie friendliness 38/100

opensearch-project/data-prepper#7022 · 2 comments · Opened Jul 24, 2026
[FEATURE] DLQ support for s3 compatible storage Open

enhancement help wanted

Difficulty 4/5 3-5 days Newbie friendliness 45/100

opensearch-project/data-prepper#7018 · 2 comments · Opened Jul 23, 2026
[RFC] GCP Secret Manager support for configuration secrets Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

opensearch-project/data-prepper#7010 · 1 comment · 1 reaction · Opened Jul 19, 2026
setuptools-78.1.1-py3-none-any.whl: 1 vulnerabilities (highest severity is: 6.1) Open

Mend: dependency security vulnerability

Difficulty 1/5 Under an hour Newbie friendliness 72/100

opensearch-project/data-prepper#7008 · Opened Jul 18, 2026
[BUG] OpenSearchSink leaks a full IOReactor (Apache HttpAsyncClient) on every AbstractSink init-retry attempt Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 72/100

opensearch-project/data-prepper#7003 · 1 comment · Opened Jul 16, 2026
SQS source plugin logs exception messages that may contain sensitive data Open

bug

opensearch-project/data-prepper#6996 · 1 assignee · Opened Jul 14, 2026
[BUG] kinesis-source-integration-tests fails on every triggered Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 55/100

opensearch-project/data-prepper#6983 · Opened Jul 5, 2026
[FEATURE] Store the source partition key as decomposed columns in the JDBC source coordination store Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 30/100

opensearch-project/data-prepper#6982 · Opened Jul 5, 2026
[Release - Action Required] Publish release for tag 2.16.0 Open

Difficulty 1/5 Under an hour Newbie friendliness 65/100

opensearch-project/data-prepper#6976 · 1 comment · Opened Jul 2, 2026
aws-cdk-lib-2.254.0.tgz: 11 vulnerabilities (highest severity is: 7.5) Open

bug Mend: dependency security vulnerability

Difficulty 2/5 1-3 hours Newbie friendliness 55/100

opensearch-project/data-prepper#6944 · Opened Jun 30, 2026
[FEATURE] Add enhanced logging for TLS handshake failures in mutual_tls authentication Open

enhancement

opensearch-project/data-prepper#6942 · 1 assignee · Opened Jun 29, 2026
[FEATURE] Support inline PEM content and multiple trust certificate files for mutual_tls Open

enhancement

opensearch-project/data-prepper#6941 · 1 assignee · Opened Jun 29, 2026
[FEATURE] Add CRL/OCSP revocation checking for mutual_tls authentication Open

enhancement

opensearch-project/data-prepper#6940 · 1 assignee · Opened Jun 29, 2026
Add endpoint-specific retryable client status support and configurable rate-limit header parsing in source-crawler Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 55/100

opensearch-project/data-prepper#6938 · Opened Jun 25, 2026
[BUG] S3-SQS source with acknowledgments does not honor shutdown_timeout Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 48/100

opensearch-project/data-prepper#6929 · Opened Jun 16, 2026
aws-cdk-lib-2.253.1.tgz: 11 vulnerabilities (highest severity is: 7.5) Open

Mend: dependency security vulnerability

Difficulty 2/5 1-3 hours Newbie friendliness 64/100

opensearch-project/data-prepper#6919 · Opened Jun 11, 2026
[BUG] NullPointerException in ExistingDocumentQueryManager.dropAndReleaseFoundEvents when query_lookup finds existing documents — documents permanently stuck, pipeline effectively blocked Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 52/100

opensearch-project/data-prepper#6902 · 1 comment · Opened Jun 3, 2026
[BUG]: No back pressure handling, if OpenSearch is temporary overloaded Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

opensearch-project/data-prepper#6901 · 2 comments · Opened Jun 1, 2026
Add mTLS client certificate authentication to HTTP-based sources Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 52/100

opensearch-project/data-prepper#6889 · 3 comments · Opened May 29, 2026
[BUG] opensearch_api source: _bulk endpoint returns empty HTTP 200 with no JSON body Open

enhancement

opensearch-project/data-prepper#6877 · 1 assignee · Opened May 21, 2026
[BUG] opensearch_api source does not work with Kafka buffer - no ByteDecoder registered Open

enhancement

opensearch-project/data-prepper#6876 · 1 assignee · Opened May 21, 2026
Pull-ingestion: Add integration tests for pull-ingestion using Kafka Open

opensearch-project/data-prepper#6840 · 1 assignee · Opened May 7, 2026
Pull-ingestion: Support end-to-end acknowledgements Open

opensearch-project/data-prepper#6839 · 1 assignee · Opened May 7, 2026
Pull-ingestion: Support pipeline DLQ Open

opensearch-project/data-prepper#6838 · 1 assignee · Opened May 7, 2026
Pull-ingestion: Auto-generate values for document Id and version Open

opensearch-project/data-prepper#6837 · 1 assignee · Opened May 7, 2026
Pull-ingestion: Create and configure the OpenSearch index automatically during initialization Open

opensearch-project/data-prepper#6836 · 1 assignee · Opened May 7, 2026
Provide a configuration for OpenSearch Data Streams Open

enhancement

opensearch-project/data-prepper#6828 · 2 reactions · 1 assignee · Opened May 6, 2026
[RFC] Pull-Based Ingestion Support in the OpenSearch Sink Open

enhancement

opensearch-project/data-prepper#6796 · 6 comments · 1 assignee · Opened May 1, 2026

opensearch-project/data-prepper

Get new issues in your inbox