DeduplicateKafkaSinkTransformer should work with fields from payload
Ouverte
enhancement
- Langage dominant
- Scala
- Étoiles
- 47
- Forks
- 14
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
When changing https://github.com/AbsaOSS/hyperdrive/blob/develop/driver/src/test/scala/za/co/absa/hyperdrive/driver/drivers/KafkaToKafkaDeduplicationAfterRetryDockerTest.scala#L73-L74
to
```
"transformer.[kafka.deduplicator].source.id.columns" -> "value.record_id",
"transformer.[kafka.deduplicator].destination.id.columns" -> "value.record_id"
```
the test fails with the following exception:
```
org.apache.spark.SparkException: Malformed records are detected in record parsing.
Caused by: org.apache.avro.AvroTypeException: Found null, expecting string
```
Even if this behavior is documented, it may never have been working.
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.