alibaba / alibaba/DataX

datax_v202205版本 odpsreader问题

Open
#1,443 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

七月初发现github datax有重大更新、下载新版本进行使用、在使用过程中、发现原本可以运行的json脚本、在新的datax运行失败、最开始报错
2022-07-12 17:22:15.164 [0-0-0-reader] ERROR RetryUtil - Exception when calling callable, 异常Msg:ErrorCode=Local Error, ErrorMessage=Specified column list is empty.
com.aliyun.odps.tunnel.TunnelException: ErrorCode=Local Error, ErrorMessage=Specified column list is empty.
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.openRecordReader(TableTunnel.java:1715) ~[odps-sdk-core-0.38.4-public.jar:0.38.4-public]
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.openRecordReader(TableTunnel.java:1693) ~[odps-sdk-core-0.38.4-public.jar:0.38.4-public]
at com.alibaba.datax.plugin.reader.odpsreader.util.OdpsUtil$7.call(OdpsUtil.java:398) ~[odpsreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.odpsreader.util.OdpsUtil$7.call(OdpsUtil.java:395) ~[odpsreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30) [datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.odpsreader.util.OdpsUtil.getRecordReader(OdpsUtil.java:395) [odpsreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.odpsreader.ReaderProxy.doRead(ReaderProxy.java:99) [odpsreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.odpsreader.OdpsReader$Task.startRead(OdpsReader.java:519) [odpsreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) [datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:750) [na:1.8.0_321]
后续开始进行重连、重连之后报以下错误
2022-07-12 17:30:46.174 [0-0-0-reader] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第9次重试.本次重试计划等待[256000]ms,实际等待[256001]ms, 异常Msg:[ErrorCode=Local E
rror, ErrorMessage=Specified column list is empty.]2022-07-12 17:30:46.175 [0-0-0-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:DATAX_R_ODPS_008:打开RecordReader失败, Solution:[请联系ODPS管理员]. - open RecordReader失败. 请联系 ODPS 管理员处理. - ErrorCode
=Local Error, ErrorMessage=Specified column list is empty. at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.openRecordReader(TableTunnel.java:1715)
at com.aliyun.odps.tunnel.TableTunnel$DownloadSession.openRecordReader(TableTunnel.java:1693)
at com.alibaba.datax.plugin.reader.odpsreader.util.OdpsUtil$7.call(OdpsUtil.java:398)
at com.alibaba.datax.plugin.reader.odpsreader.util.OdpsUtil$7.call(OdpsUtil.java:395)
at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164)
at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111)
at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30)
at com.alibaba.datax.plugin.reader.odpsreader.util.OdpsUtil.getRecordReader(OdpsUtil.java:395)
at com.alibaba.datax.plugin.reader.odpsreader.ReaderProxy.doRead(ReaderProxy.java:99)
at com.alibaba.datax.plugin.reader.odpsreader.OdpsReader$Task.startRead(OdpsReader.java:519)
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
at java.lang.Thread.run(Thread.java:750)
json脚本应该没有问题的 、原先的版本是可以运行的、希望有大佬能够答惑、谢谢谢谢!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the older working DataX version with datax_v202205 and tracing the supplied failure through odpsreader.util.OdpsUtil.getRecordReader, ReaderProxy.doRead, and OdpsReader$Task.startRead. Use the failing JSON script and the ODPS stack trace to reproduce the error. Done means identifying the version-related cause of the empty column list and documenting or fixing the reader behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.