Add support for slowly changing mapping tables
オープン
Conformance
feature
priority: medium
under discussion
- 主要言語
- Scala
- スター
- 33
- フォーク
- 16
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
## Background
Streaming conformance currently uses static mapping tables read from HDFS.
If a job is running long enough the mapping tables become outdated.
## Feature
Long-running streaming conformance jobs need a way to make in-memory mapping tables up to date.
Not sure yet how to pick up the changes to mapping tables.
- [ ] Periodically check the HDFS folder with mapping tables for updates
- [ ] Subscribe to a Kafka topic with changes to mapping tables
Which way to go depends on how MDM and MDR are done.
コントリビューションガイド
評価
この issue はまだ評価されていません。