Aiven-Open / Aiven-Open/metadata-parser

[EPIC] Apache Kafka Connect Connectors

Abierto
#10 0 comentarios 0 reacciones 0 asignados Ver en GitHub
help wanted
Lenguaje dominante
Python
Estrellas
14
Forks
7
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

# What is currently missing?

The Apache Kafka Connect Connector parsing is tedious since we need to parse the JSON definition for each of them. As of now the basic Debezium PG source and OpenSearch sink are done but can be improved. Other connectors (see `explore_kafka_connect` in `explore_service.py`) are:

**Source**

* [ ] [Couchbase](https://github.com/couchbase/kafka-connect-couchbase)
* [ ] [Official MongoDB](https://docs.mongodb.com/kafka-connector/current/) preview
* [ ] [Debezium for MongoDB](https://debezium.io/docs/connectors/mongodb/)
* [ ] [Debezium for MySQL](https://debezium.io/docs/connectors/mysql/)
* [X] [Debezium for PostgreSQL](https://developer.aiven.io/docs/products/kafka/kafka-connect/howto/debezium-source-connector-pg.html)
* [ ] [Debezium for SQL Server](https://debezium.io/docs/connectors/sqlserver/)
* [ ] [Google Cloud Pub/Sub](https://github.com/GoogleCloudPlatform/pubsub/tree/master/kafka-connector)
* [ ] [Google Cloud Pub/Sub Lite](https://github.com/GoogleCloudPlatform/pubsub/)
* [ ] [JDBC](https://github.com/aiven/aiven-kafka-connect-jdbc/blob/master/docs/source-connector.md)
* [ ] Schema Source
* [ ] [Stream Reactor Cassandra](https://docs.lenses.io/connectors/source/cassandra.html)
* [ ] [Stream Reactor MQTT](https://docs.lenses.io/connectors/source/mqtt.html)

**Sink**

* [ ] [Aiven for Apache Kafka S3 Sink Connector](https://developer.aiven.io/docs/products/kafka/kafka-connect/howto/s3-sink-connector-aiven.html)
* [ ] [Confluent Amazon S3 Sink](https://help.aiven.io/kafka/aiven-kafka-kafka-connect-s3)
* [ ] [Couchbase](https://github.com/couchbase/kafka-connect-couchbase)
* [ ] [Elasticsearch](https://help.aiven.io/kafka/aiven-kafka-elasticsearch-sink-connector)
* [ ] [Google BigQuery](https://github.com/wepay/kafka-connect-bigquery)
* [ ] [Google Cloud Pub/Sub](https://github.com/GoogleCloudPlatform/pubsub/)
* [ ] [Google Cloud Pub/Sub Lite](https://github.com/GoogleCloudPlatform/pubsub/)
* [ ] [Google Cloud Storage](https://help.aiven.io/kafka/connectors/aiven-kafka-gcs-sink-connector)
* [ ] [HTTP](https://github.com/aiven/aiven-kafka-connect-http) preview
* [ ] [JDBC](https://github.com/aiven/aiven-kafka-connect-jdbc/blob/master/docs/sink-connector.md)
* [ ] [Official MongoDB](https://docs.mongodb.com/kafka-connector/current/) preview
* [X] [OpenSearch](https://github.com/aiven/opensearch-connector-for-apache-kafka/blob/main/docs/opensearch-sink-connector-config-options.rst) preview
* [ ] [Snowflake](https://docs.snowflake.net/manuals/user-guide/kafka-connector.html) preview
* [ ] [Splunk](https://github.com/splunk/kafka-connect-splunk)
* [ ] [Stream Reactor Cassandra](https://docs.lenses.io/connectors/sink/cassandra.html)
* [ ] [Stream Reactor InfluxDB](https://docs.lenses.io/connectors/sink/influx.html)
* [ ] [Stream Reactor Mongo DB](https://docs.lenses.io/connectors/sink/mongo.html)
* [ ] [Stream Reactor MQTT](https://docs.lenses.io/connectors/sink/mqtt.html)
* [ ] [Stream Reactor Redis](https://docs.lenses.io/connectors/sink/redis.html)

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.