Aiven-Open / Aiven-Open/bigquery-connector-for-apache-kafka
Config enableBatchLoad does not work correctly when original topic name was mutated in SMT
- Dominant language
- Java
- Stars
- 37
- Forks
- 45
- Avg merge
- 19h 50m
- Merged PRs (30d)
- 5
Description
Following [my previous PR](https://github.com/Aiven-Open/bigquery-connector-for-apache-kafka/pull/70), when `enableBatchLoad` config is set, it doesn't work correctly when the topic name was mutated in `SinkRecord`.
I will create a separate PR if I have time to work on this. If you have any thoughts on implementation (for example, should we make the same handle for `SinkRecord.originalTopic` or approach it differently), please let me know. Ref to the problematic code is [here](https://github.com/Aiven-Open/bigquery-connector-for-apache-kafka/blob/0142bdac1bddbac74188715b0a5d984b8348fb7e/kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/BigQuerySinkTask.java#L300).
By the way, what is the use case for having `enableBatchLoad` as list? As a user, I would expect this to be a boolean value, applied to all topics handled by the connector, not a subset - that would be much more convenient.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.