alibaba / alibaba/DataX

RdbmsWriter Date conversion depends on system default timezone, timezone settings in conversion methods not work

Open
#1,325 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

Description same as the subject

I encountered a problem that all date type values written into MySQL had been subtracted one day

Then I checked the system time zone, as bellow

```shell
$ date -R
Mon, 21 Mar 2022 17:38:36 +0800

$ date +%Z
CST

$ ll /etc/localtime
lrwxrwxrwx 1 root root 25 Oct 1 2019 /etc/localtime -> ../usr/share/zoneinfo/UTC
```

related implementation in DataX

https://github.com/alibaba/DataX/blob/ef8459376072b66c252235fb41e326b88105fb45/plugin-rdbms-util/src/main/java/com/alibaba/datax/plugin/rdbms/writer/CommonRdbmsWriter.java#L460

https://github.com/alibaba/DataX/blob/ef8459376072b66c252235fb41e326b88105fb45/common/src/main/java/com/alibaba/datax/common/element/ColumnCast.java#L78

https://github.com/alibaba/DataX/blob/ef8459376072b66c252235fb41e326b88105fb45/common/src/main/java/com/alibaba/datax/common/element/ColumnCast.java#L98

It looks like the timezone settings in FastDateFormat does not work

### Ref:

* [Calendar.getInstance(TimeZone.getTimeZone("UTC")) is not returning UTC time](https://stackoverflow.com/questions/21349475/calendar-getinstancetimezone-gettimezoneutc-is-not-returning-utc-time)
* [Converting string to date with timezone](https://stackoverflow.com/questions/4203718/converting-string-to-date-with-timezone)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked conversion code in plugin-rdbms-util/src/main/java/com/alibaba/datax/plugin/rdbms/writer/CommonRdbmsWriter.java and common/src/main/java/com/alibaba/datax/common/element/ColumnCast.java. Reproduce writing date values when the system timezone differs from the configured timezone, then trace the FastDateFormat and Calendar behavior. Done means MySQL date values are not shifted by a day and the timezone settings used by the conversion methods take effect.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.