feat(connectors): Iggy source connector implementation
- Dominant language
- Rust
- Stars
- 4.9k
- Forks
- 432
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 173
Description
Parent issue: #3764
Implement a source connector module that consumes messages from an upstream Iggy (cluster) into the connectors runtime. This issue serves as the overall implementation tracker. Please refer to the original proposal for detailed design.
## Milestones
### M1: Basic functionality
- Messages flow from upstream Iggy (source) through the connector to downstream Iggy
- Configurable stream and topic selection
- Progress persistence with resumable offset tracking
### M2: Fine-grained configuration
- TLS support
- Additional authentication methods (username/password, token)
- Flow control: finer-grained concurrency control and batching
### M3: Enhancement
- Compression and pro-active retry
- Partition-aware offset tracking
Contributor guide
Research direction
Start with parent issue #3764 and the original proposal, since this tracker names no files, tests, or entry points. Map the connectors runtime and the upstream Iggy source flow, then use the milestones as the completion checklist: message flow, selection, resumable offsets, followed by TLS, authentication, flow control, compression, retries, and partition-aware tracking.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100