alibaba / alibaba/DataX

脏数据导致任务停止

Open
#583 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

报错信息:
2020-01-19 19:56:07.387 [0-1-145-reader] ERROR StdoutPluginCollector - 脏数据:
{"exception":"Connection reset","record":[{"byteSize":37,"index":0,"rawData":"zhongqingtengyuzhiye_company26fd0ce1b","type":"STRING"},{"byteSize":7,"index":1,"rawData":"3367620","type":"DOUBLE"},{"byteSize":6,"index":2,"rawData":"169960","type":"DOUBLE"},{"byteSize":8,"index":3,"rawData":"CUSTOMER","type":"STRING"},{"byteSize":3,"index":4,"rawData":"449","type":"DOUBLE"},{"byteSize":8,"index":5,"rawData":1553059632107,"type":"DATE"},{"byteSize":2,"index":6,"rawData":"去电","type":"STRING"}],"type":"reader"}
2020-01-19 19:56:07.389 [0-1-145-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-06], Description:[执行数据库 Sql 失败, 请检查您的配置的 column/table/where/querySql或者向 DBA 寻求帮助.]. - 执行的SQL为: select companyuuid='abc_company26fd0ce1b',[followid],[customerid],[followcategory],[userid],[followcreatedtime],[followtype],[contents],[enablealert],[alerttime],[alertdeptid],[alertuserid],[alertcontents],[deleted],[createduserid],[createdtime],[updateduserid],[updatedtime],[followuuid],[followtypeid],[addtionalbusinessid],[uniqueid],[useruuid],[alertdeptuuid],[alertuseruuid],[createduseruuid],[updateduseruuid],[deptid],[customeremployeeid1],[customerdeptid1],[writetype] from fxt_web_zhongqingtengyuzhiye.dbo.pro_customer_follow with(nolock) 具体错误信息为:com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:26) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.util.RdbmsException.asQueryException(RdbmsException.java:97) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Task.startRead(CommonRdbmsReader.java:220) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.sqlserverreader.SqlServerReader$Task.startRead(SqlServerReader.java:79) ~[sqlserverreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]
**显示为脏数据的这条数据重新导入时又成功了,数据完全没有变过,但是这个表经常报这种错,请问这种问题怎么解决,如何排查出错原因**

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the reported SQL Server logs and the reader entry points in CommonRdbmsReader.java, SqlServerReader.java, and ReaderRunner.java. Reproduce the intermittent “The connection is closed” failure and trace how the connection error becomes dirty data and stops the task. Done means the failure cause is identified and the issue defines a tested handling or recovery change.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.