[Bug] Iceberg Source OR Doris Sink time zone issue
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 203
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
Hi, when I try to use Spark to sync data from Iceberg to Doris, I found a timezone issue with `TIMESTAMP` fields.
Spark timezone:
```
spark-sql> SET spark.sql.session.timeZone;
spark.sql.session.timeZone Asia/Shanghai
```
Doris timezone:
```
SHOW VARIABLES LIKE 'time_zone';
| Variable_name | Value | Default_Value | Changed |
| time_zone | Asia/Shanghai | Asia/Shanghai | 0 |
```
However, the actual data written into Doris is in UTC time (i.e. offset by -8 hours).
I also tried setting the timezone in the environment config, but it does not work, for example:
```
env {
parallelism = 1
job.mode = "BATCH"
spark.sql.session.timeZone = "Etc/UTC"
}
```
But it still does not take effect.
How can I fix this issue?
### SeaTunnel Version
2.3.13
### SeaTunnel Config
```conf
env {
parallelism = 1
job.mode = "BATCH"
}
source {
Iceberg {
catalog_name = "${catalog}"
iceberg.catalog.config={
type = "rest"
warehouse = "${warehouse}"
uri = "${rest.uri}"
header.X-Iceberg-Access-Delegation = "vended-credentials"
}
namespace = "${source.iceberg.database}"
table = "xxx"
query = "SELECT xxx FROM xxx WHERE dt = '${etldate}'"
plugin_output = "iceberg"
}
}
sink {
Doris {
fenodes = "${sink.doris.fenodes}"
username = "${sink.doris.username}"
password = "${sink.doris.password}"
database = "${sink.doris.database}"
table = "xxx"
sink.label-prefix = "xxx"
data_save_mode = "CUSTOM_PROCESSING"
custom_sql = "xxx"
doris.config {
format = "json"
read_json_by_line = "true"
}
}
}
```
### Running Command
```shell
${SPARK_HOME}/bin/spark-submit --class "org.apache.seatunnel.core.starter.spark.SeaTunnelSpark" --name "SeaTunnel" --master "local" --deploy-mode "client" --jars "/opt/soft/seatunnel-2.3.13/lib/seatunnel-hadoop3-3.1.4-uber.jar,/opt/soft/seatunnel-2.3.13/lib/seatunnel-hadoop-aws.jar,/opt/soft/seatunnel-2.3.13/lib/seatunnel-transforms-v2.jar,/opt/soft/seatunnel-2.3.13/lib/mysql-connector-j-8.2.0.jar,/opt/soft/seatunnel-2.3.13/connectors/connector-iceberg-2.3.13.jar,/opt/soft/seatunnel-2.3.13/connectors/connector-doris-2.3.13.jar"
```
### Error Exception
```log
nothing happend
```
### Zeta or Flink or Spark Version
spark-3.3.4
### Java or Scala Version
jdk-11
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.