embulk / embulk/embulk-input-jdbc

[MySQL] last_record with timestamp column stored wrong timestamp.

Open
#126 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
106
Forks
76
PR merge metrics
No merged PRs in 30d

Description

* Embulk: 0.8.39
* embulk-input-mysql: 0.8.2
* MySQL: 5.7.19
* Server and Client timezone: JST.
* sample_data: sample.txt

Those data is `JST` timezone.

```
mysql> select * from in_mysql_test;
+------+---------------------+-------+
| c0 | c1 | c2 |
+------+---------------------+-------+
| 1 | 2017-12-01 00:00:00 | dummy |
| 2 | 2017-12-01 00:01:00 | dummy |
...
| 1440 | 2017-12-01 23:59:00 | dummy |
| 1441 | 2017-12-02 00:00:00 | dummy |
+------+---------------------+-------+

```

```yaml
in:
type: mysql
host: 127.0.0.1
user: {{ env.mysql_user }}
password: {{ env.mysql_password }}
table: in_mysql_test
database: embulk_test
options: { useLegacyDatetimeCode: false }
incremental_columns:
- c1
incremental: true

out:
type: stdout
```

Execute `embulk run -o diff.yml input_mysql.yml.liquid`

```
in:
last_record: ['2017-12-01T06:00:00.000000Z']
out: {}
```

```
1439,2017-12-01 14:58:00,dummy
1440,2017-12-01 14:59:00,dummy
1441,2017-12-01 15:00:00,dummy
2017-12-07 17:31:58.364 +0900 [INFO] (0001:transaction): {done: 1 / 1, running: 0}
2017-12-07 17:31:58.370 +0900 [INFO] (main): Committed.
2017-12-07 17:31:58.371 +0900 [INFO] (main): Next config diff: {"in":{"last_record":["2017-12-01T06:00:00.000000Z"]},"out":{}}
```

The `last_record` should be `2017-12-01T15:00:00.000000Z` but It set `2017-12-01T06:00:00.000000Z`

And second execution log

```
2017-12-07 20:05:33.310 +0900 [INFO] (0018:task-0000): SQL: SELECT * FROM `in_mysql_test` WHERE ((`c1` > ?)) ORDER BY `c1`
2017-12-07 20:05:33.311 +0900 [INFO] (0018:task-0000): Parameters: ["2017-12-01T06:00:00.000000Z"]
```

## Reference

* [Reproduce codes](https://github.com/hiroyuki-sato/embulk-support/tree/master/twitter_in_mysql_last_record)
* [Twitter](https://twitter.com/ganchiku/status/938590077176840193)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.