Streaming to kafka
- Dominant language
- Java
- Stars
- 3.7k
- Forks
- 736
- PR merge metrics
- No merged PRs in 30d
Description
I know it's a long shot, but asking anyway incase someone has already solved this.
Can I pump the delta to kafka instead of keeping it in memory and waiting for an agent to poll the endpoint? is it possible with some tweaks?
My usecase - An Open source GoldenGate equivalent which will stream all the DML operation in the source Oracle to a compacted kafka topic.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the existing delta delivery path, including the endpoint agents poll and where deltas are held in memory. Define the required behavior for streaming source Oracle DML operations to a compacted Kafka topic instead of waiting for agent polling; the issue does not identify files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100