Support Full load and CDC from AWS Neptune
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- aws
- Domain
- data-engineering, databases
Research direction
Start with the Neptune export and Neptune Streams documentation linked in the issue, then inspect the repository's existing source plugins and OpenSearch sink integration points. Define how full export and CDC events should reach the sink, and verify both ingestion paths with appropriate integration coverage.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
I would like to use data-prepper to export AWS Neptune graph database data and also ingest stream data changes from Neptune to OpenSearch Service domains and collections.
Describe the solution you'd like
Support a new Neptune source plugin to
- integrate with
neptune-exporttool to export the entire Neptune graph database to Opensearch Sink. - read the Neptune stream data and ingest any change data capture events to Opensearch Sink.
Additional context
- Neptune export tool https://docs.aws.amazon.com/neptune/latest/userguide/neptune-export.html
- Neptune Streams https://docs.aws.amazon.com/neptune/latest/userguide/streams.html
- Dominant language
- Java
- Stars
- 374
- Forks
- 355
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from opensearch-project/data-prepper
-
[BUG] Kafka sink throws NullPointerException on every shutdown when thread_wait_time is not set Openbug untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
opensearch-project/data-prepper#7130 ·
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
opensearch-project/data-prepper#7112 ·
-
Mend: dependency security vulnerability
Difficulty 1/5 Under an hour Newbie friendliness 72/100
opensearch-project/data-prepper#7008 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 65/100
opensearch-project/data-prepper#6976 · 1 comment ·
-
Mend: dependency security vulnerability
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
opensearch-project/data-prepper#6919 ·
All issues in opensearch-project/data-prepper
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
bug needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
objectionary/hone-maven-plugin#1061 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
spring-projects/spring-modulith#1895 ·