embulk / embulk/embulk-output-jdbc

embulk-output-mysql (0.8.7) TimeZone(Unknown Source)

Open
#308 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
92
Forks
89
PR merge metrics
No merged PRs in 30d

Description

Hi All,
I'm using:
● MySQL: 5.7.14
● embulk: 0.9.23
● embulk-output-mysql: 0.8.7

below is my config
in:
type: http
url: https://xxxx
parser:
type: xml2
root: aaa/bbb
schema:
- {name: tsr_id, type: long}
- {name: date, type: timestamp, format: '%Y/%m/%d'}}
method: get
read_timeout: 300000
out:
type: mysql
host: xxx.xxx.xxx.xxx
user: user
password: *******
database: db
table: table_xxx
options: {useLegacyDatetimeCode: false, serverTimezone: UTC}
mode: insert
column_options:
date: {type: 'DATETIME'}

however, sometimes i get error about time zone
below is the log I got

`java.io.IOException: Stream closed
at java.util.zip.InflaterInputStream.ensureOpen(InflaterInputStream.java:67)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:121)
at java.io.FilterInputStream.read(FilterInputStream.java:83)
at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:288)
at org.joda.time.tz.DateTimeZoneBuilder.readMillis(DateTimeZoneBuilder.java:183)
at org.joda.time.tz.DateTimeZoneBuilder$PrecalculatedZone.readFrom(DateTimeZoneBuilder.java:1359)
at org.joda.time.tz.DateTimeZoneBuilder.readFrom(DateTimeZoneBuilder.java:119)
at org.joda.time.tz.DateTimeZoneBuilder.readFrom(DateTimeZoneBuilder.java:98)
at org.joda.time.tz.ZoneInfoProvider.loadZoneData(ZoneInfoProvider.java:226)
at org.joda.time.tz.ZoneInfoProvider.getZone(ZoneInfoProvider.java:160)
at org.joda.time.DateTimeZone.forID(DateTimeZone.java:216)
at org.joda.time.DateTimeUtils.put(DateTimeUtils.java:451)
at org.joda.time.DateTimeUtils.buildDefaultTimeZoneNames(DateTimeUtils.java:441)
at org.joda.time.DateTimeUtils.getDefaultTimeZoneNames(DateTimeUtils.java:412)
at org.joda.time.format.DateTimeFormatterBuilder$TimeZoneName.parseInto(DateTimeFormatterBuilder.java:2294)
at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:179)
at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:780)
at org.embulk.spi.time.TimeZoneIds.parseJodaDateTimeZone(TimeZoneIds.java:142)
at org.embulk.spi.time.TimestampFormatter$TimestampColumnOption.getTimeZone(TimestampFormatter.java:130)
at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
at org.embulk.config.TaskInvocationHandler.invoke(TaskInvocationHandler.java:165)
at org.embulk.output.jdbc.setter.$Proxy53.getTimeZone(Unknown Source)
at org.embulk.spi.time.TimestampFormatter.(TimestampFormatter.java:29)
at org.embulk.output.jdbc.setter.ColumnSetterFactory.newTimestampFormatter(ColumnSetterFactory.java:92)
at org.embulk.output.jdbc.setter.ColumnSetterFactory.newCoalesceColumnSetter(ColumnSetterFactory.java:143)
at org.embulk.output.jdbc.setter.ColumnSetterFactory.newColumnSetter(ColumnSetterFactory.java:42)
at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.newColumnSetters(AbstractJdbcOutputPlugin.java:787)
at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.doBegin(AbstractJdbcOutputPlugin.java:595)
at org.embulk.output.MySQLOutputPlugin.doBegin(MySQLOutputPlugin.java:164)
at org.embulk.output.jdbc.AbstractJdbcOutputPlugin$2.run(AbstractJdbcOutputPlugin.java:452)
at org.embulk.output.jdbc.AbstractJdbcOutputPlugin$RetryableSQLExecution.call(AbstractJdbcOutputPlugin.java:1284)
at org.embulk.output.jdbc.AbstractJdbcOutputPlugin$RetryableSQLExecution.call(AbstractJdbcOutputPlugin.java:1272)
at org.embulk.spi.util.RetryExecutor.run(RetryExecutor.java:81)
at org.embulk.spi.util.RetryExecutor.runInterruptible(RetryExecutor.java:62)
at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.withRetry(AbstractJdbcOutputPlugin.java:1257)
at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.withRetry(AbstractJdbcOutputPlugin.java:1249)
at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.begin(AbstractJdbcOutputPlugin.java:446)
at org.embulk.output.jdbc.AbstractJdbcOutputPlugin.transaction(AbstractJdbcOutputPlugin.java:422)
at org.embulk.exec.BulkLoader$4$1$1.transaction(BulkLoader.java:521)
at org.embulk.exec.LocalExecutorPlugin.transaction(LocalExecutorPlugin.java:50)
at org.embulk.exec.BulkLoader$4$1.run(BulkLoader.java:516)
at org.embulk.spi.util.Filters$RecursiveControl.transaction(Filters.java:84)
at org.embulk.spi.util.Filters.transaction(Filters.java:42)
at org.embulk.exec.BulkLoader$4.run(BulkLoader.java:511)
at org.embulk.input.jdbc.AbstractJdbcInputPlugin.transaction(AbstractJdbcInputPlugin.java:225)
at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:507)
at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:35)
at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:353)
at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:350)
at org.embulk.spi.Exec.doWith(Exec.java:22)
at org.embulk.exec.BulkLoader.run(BulkLoader.java:350)
at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:242)
at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:291)
at org.embulk.EmbulkRunner.run(EmbulkRunner.java:155)
at org.embulk.cli.EmbulkRun.runSubcommand(EmbulkRun.java:431)
at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:90)
at org.embulk.cli.Main.main(Main.java:64)
`

Can someone help me solve this problem? thanks!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the supplied configuration and stack trace, especially MySQLOutputPlugin.doBegin, ColumnSetterFactory.newTimestampFormatter, and TimestampFormatter.getTimeZone. Reproduce the intermittent failure with Embulk 0.9.23 and embulk-output-mysql 0.8.7, then identify whether the timezone-loading error is reproducible and document a verified resolution or minimal reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.