Aiven-Open / Aiven-Open/bigquery-connector-for-apache-kafka

Debezium DATETIME type ingested as TIMESTAMP adding unwanted timezone semantics

未關閉
#184 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
enhancement
主要語言
Java
星號
37
分支
45
平均合併
19 小時 50 分鐘
30 天內合併 PR
5

描述

## Problem Summary
When using Debezium as a data source with the BigQuery Sink connector, timezone-agnostic `DATETIME` data from the source database is being incorrectly mapped to BigQuery's UTC-fixed TIMESTAMP type.

## Details
### Expected Behavior
The connector should map Debezium's `io.debezium.time.Timestamp` and `io.debezium.time.MicroTimestamp` logical types to BigQuery's native `DATETIME` type. This preserves the timezone-agnostic "local" semantics of the source data.

### Actual Behavior
The connector currently maps these logical types to BigQuery's `TIMESTAMP` type. Since BigQuery `TIMESTAMP` is always stored as a UTC point-in-time, this forces an implicit UTC offset onto the data.

### Business Impact
This could result in a mismatch, where local time values appear shifted when displayed or queried in different timezones. Users would be unable to maintain local time representation from their source database within BigQuery.

### Potential Cause
In [DebeziumLogicalConverters.java](https://github.com/Aiven-Open/bigquery-connector-for-apache-kafka/blob/bc35e64fcde7cc3e227561a99102669289092fef/kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/convert/logicaltype/DebeziumLogicalConverters.java), both the TimestampConverter and MicroTimestampConverter are explicitly hardcoded to use `LegacySQLTypeName.TIMESTAMP` ([1](https://github.com/Aiven-Open/bigquery-connector-for-apache-kafka/blob/bc35e64fcde7cc3e227561a99102669289092fef/kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/convert/logicaltype/DebeziumLogicalConverters.java#L201-L205), [2](https://github.com/Aiven-Open/bigquery-connector-for-apache-kafka/blob/bc35e64fcde7cc3e227561a99102669289092fef/kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/convert/logicaltype/DebeziumLogicalConverters.java#L148-L152)). There is currently no configuration option to override this mapping to use the `DATETIME` type.

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。