agile-lab-dev / agile-lab-dev/darwin

Kafka connector

未關閉
#66 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
contest good first issue
主要語言
Scala
星號
36
分支
10
PR 合併指標
30 天內沒有已合併 PR

描述

Do you think that make sense to have a Kafka connector? Store schemas directly in Kafka?

We can have a topic with infinite retention, where the key is the schema id and the value is the schema (plus other info if?).
This connector can write to this topic and maintain a list of id->schema in memory by consuming its own the records.
Of course the assumption is that we don't have milions of schemas...

Each instance of this connector reads and maintain all the schemas in memory, so each instance is indipendent from the other.
At every startup it reads the topic from the start.

The main benefit is that we don't have external dependencies for use cases where we just use Kafka and that we receive new schemas immediately.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。