apache / apache/iggy

feat: MySQL Connector (Source + Sink)

Open
#3,445 0 comments 0 reactions 1 assignee Claimed by @tusharagrahari View on GitHub
connectors
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

### Description

Apache Iggy should support MySQL as a connector for both
Source and Sink.

MySQL is one of the most widely used relational databases.
Supporting it as a connector would allow users to either
ingest data from MySQL into Iggy streams or push Iggy
stream data into MySQL tables.

Affected area / component: Connectors

Proposed solution
Add `core/connectors/sinks/mysql_sink` and
`core/connectors/sources/mysql_source`.

I'll start with the Source implementation first and follow
up with sink as a separate PR.

Expected support:
* Remote MySQL endpoint configuration
* Credential fields stored as SecretString
* Configurable table target
* Insert and upsert modes
* Payload formats: json, text, raw
* Configurable batch size and retry config
* Follow existing connector layout and naming conventions

Note: Will study existing connectors like postgres_source
as reference and open a discussion if any design questions
come up during implementation.

### Affected area / component

Connectors

### Proposed solution

_No response_

### Alternatives considered

_No response_

### Contribution

- [x] I'm willing to submit a pull request to implement this feature

### Good first issue

- [ ] I think this could be a good first issue for a new contributor

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.