confluentinc / confluentinc/examples

Consider adding Storage Connectors

Open
#25 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

I see there are examples for JDBC and Elastic, so perhaps the storage based connectors can be included as well?

HDFS: A Hadoop installation shouldn't be required since HDFS URI can be set to local filesystem. I've tested with `hdfs.url=file:///tmp/kafka-connect`, and it worked as expected.

S3: An AWS account isn't required. See - https://blog.minio.io/journaling-kafka-messages-with-s3-connector-and-minio-83651a51045d

I have some workable code for at least S3 here - https://github.com/cricket007/kafka-connect-sandbox

---

Extras: Use something like Apache Drill to connect to the local Zookeeper Server and run Drill SQL queries over data (can query JSON and Avro files)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.