AbsaOSS / AbsaOSS/hyperdrive

Rename prefix for additional properties for KafkaStreamReader

Open
#91 1 comment 1 reaction 1 assignee Assigned to @kevinwallimann View on GitHub
breaking-changes
Dominant language
Scala
Stars
47
Forks
14
PR merge metrics
No merged PRs in 30d

Description

Currently, additional properties for the `KafkaStreamReader` have to be specified with the prefix `reader.options.` E.g. `reader.options.kafka.security.protocol` or `reader.options.kafka.ssl.key.password`

This prefix is inconsistent with all other properties which start with `decoder.avro.`, `writer.parquet.`, `manager.checkpoint` or `transformer.columns.`. The properties for the `KafkaStreamReader` should start with `reader.kafka.`, i.e. `reader.kafka.options.kafka.security.protocol`.

For additional properties like `reader.options.failOnDataLoss` or `reader.options.minPartitions` it's hard to find out which reader implementation they belong to.

**Tasks**
- Change prefix for additional properties for `KafkaStreamReader` to `reader.kafka.options.`

**How to migrate**
- All property keys starting with `reader.options` have to be replaced by `reader.kafka.options`

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.