Aiven-Open / Aiven-Open/cloud-storage-connectors-for-apache-kafka
What happens if first record does not have headers but further records in the same batch do?
- 主要语言
- Java
- 星标
- 58
- 派生
- 39
- 平均合并
- 2 天 10 小时
- 30 天内合并 PR
- 5
描述
Output schema (avro/parquet) is based on the first record. This works fine for key/value, etc. but headers may differ between records. Different record header elements types can cause issues but this can be solved by casting header types by using StringConverter or similar for headers.
Though, when first record does not have headers the schema type is null. We should figure out what happens with next records in the same batch if they do have header (are the headers missing? does conversion fail?) and provide an approach on how to process infer the schema for headers properly.
See https://github.com/Aiven-Open/gcs-connector-for-apache-kafka/issues/347#issuecomment-1964537795
贡献指南
评估
这个 Issue 还没有评估数据。