linkedin / linkedin/databus

Streaming to kafka

Open
#123 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
3.7k
Forks
736
PR merge metrics
No merged PRs in 30d

Description

I know it's a long shot, but asking anyway incase someone has already solved this.
Can I pump the delta to kafka instead of keeping it in memory and waiting for an agent to poll the endpoint? is it possible with some tweaks?

My usecase - An Open source GoldenGate equivalent which will stream all the DML operation in the source Oracle to a compacted kafka topic.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the existing delta delivery path, including the endpoint agents poll and where deltas are held in memory. Define the required behavior for streaming source Oracle DML operations to a compacted Kafka topic instead of waiting for agent polling; the issue does not identify files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka
Domain
data-engineering, stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.