Aiven-Open / Aiven-Open/metadata-parser

[EPIC] Apache Kafka Connect Connectors

Ouverte
#10 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
help wanted
Langage dominant
Python
Étoiles
14
Forks
7
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

# What is currently missing?

The Apache Kafka Connect Connector parsing is tedious since we need to parse the JSON definition for each of them. As of now the basic Debezium PG source and OpenSearch sink are done but can be improved. Other connectors (see `explore_kafka_connect` in `explore_service.py`) are:

**Source**

* [ ] [Couchbase](https://github.com/couchbase/kafka-connect-couchbase)
* [ ] [Official MongoDB](https://docs.mongodb.com/kafka-connector/current/) preview
* [ ] [Debezium for MongoDB](https://debezium.io/docs/connectors/mongodb/)
* [ ] [Debezium for MySQL](https://debezium.io/docs/connectors/mysql/)
* [X] [Debezium for PostgreSQL](https://developer.aiven.io/docs/products/kafka/kafka-connect/howto/debezium-source-connector-pg.html)
* [ ] [Debezium for SQL Server](https://debezium.io/docs/connectors/sqlserver/)
* [ ] [Google Cloud Pub/Sub](https://github.com/GoogleCloudPlatform/pubsub/tree/master/kafka-connector)
* [ ] [Google Cloud Pub/Sub Lite](https://github.com/GoogleCloudPlatform/pubsub/)
* [ ] [JDBC](https://github.com/aiven/aiven-kafka-connect-jdbc/blob/master/docs/source-connector.md)
* [ ] Schema Source
* [ ] [Stream Reactor Cassandra](https://docs.lenses.io/connectors/source/cassandra.html)
* [ ] [Stream Reactor MQTT](https://docs.lenses.io/connectors/source/mqtt.html)

**Sink**

* [ ] [Aiven for Apache Kafka S3 Sink Connector](https://developer.aiven.io/docs/products/kafka/kafka-connect/howto/s3-sink-connector-aiven.html)
* [ ] [Confluent Amazon S3 Sink](https://help.aiven.io/kafka/aiven-kafka-kafka-connect-s3)
* [ ] [Couchbase](https://github.com/couchbase/kafka-connect-couchbase)
* [ ] [Elasticsearch](https://help.aiven.io/kafka/aiven-kafka-elasticsearch-sink-connector)
* [ ] [Google BigQuery](https://github.com/wepay/kafka-connect-bigquery)
* [ ] [Google Cloud Pub/Sub](https://github.com/GoogleCloudPlatform/pubsub/)
* [ ] [Google Cloud Pub/Sub Lite](https://github.com/GoogleCloudPlatform/pubsub/)
* [ ] [Google Cloud Storage](https://help.aiven.io/kafka/connectors/aiven-kafka-gcs-sink-connector)
* [ ] [HTTP](https://github.com/aiven/aiven-kafka-connect-http) preview
* [ ] [JDBC](https://github.com/aiven/aiven-kafka-connect-jdbc/blob/master/docs/sink-connector.md)
* [ ] [Official MongoDB](https://docs.mongodb.com/kafka-connector/current/) preview
* [X] [OpenSearch](https://github.com/aiven/opensearch-connector-for-apache-kafka/blob/main/docs/opensearch-sink-connector-config-options.rst) preview
* [ ] [Snowflake](https://docs.snowflake.net/manuals/user-guide/kafka-connector.html) preview
* [ ] [Splunk](https://github.com/splunk/kafka-connect-splunk)
* [ ] [Stream Reactor Cassandra](https://docs.lenses.io/connectors/sink/cassandra.html)
* [ ] [Stream Reactor InfluxDB](https://docs.lenses.io/connectors/sink/influx.html)
* [ ] [Stream Reactor Mongo DB](https://docs.lenses.io/connectors/sink/mongo.html)
* [ ] [Stream Reactor MQTT](https://docs.lenses.io/connectors/sink/mqtt.html)
* [ ] [Stream Reactor Redis](https://docs.lenses.io/connectors/sink/redis.html)

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.