脏数据导致任务停止
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
报错信息:
2020-01-19 19:56:07.387 [0-1-145-reader] ERROR StdoutPluginCollector - 脏数据:
{"exception":"Connection reset","record":[{"byteSize":37,"index":0,"rawData":"zhongqingtengyuzhiye_company26fd0ce1b","type":"STRING"},{"byteSize":7,"index":1,"rawData":"3367620","type":"DOUBLE"},{"byteSize":6,"index":2,"rawData":"169960","type":"DOUBLE"},{"byteSize":8,"index":3,"rawData":"CUSTOMER","type":"STRING"},{"byteSize":3,"index":4,"rawData":"449","type":"DOUBLE"},{"byteSize":8,"index":5,"rawData":1553059632107,"type":"DATE"},{"byteSize":2,"index":6,"rawData":"去电","type":"STRING"}],"type":"reader"}
2020-01-19 19:56:07.389 [0-1-145-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-06], Description:[执行数据库 Sql 失败, 请检查您的配置的 column/table/where/querySql或者向 DBA 寻求帮助.]. - 执行的SQL为: select companyuuid='abc_company26fd0ce1b',[followid],[customerid],[followcategory],[userid],[followcreatedtime],[followtype],[contents],[enablealert],[alerttime],[alertdeptid],[alertuserid],[alertcontents],[deleted],[createduserid],[createdtime],[updateduserid],[updatedtime],[followuuid],[followtypeid],[addtionalbusinessid],[uniqueid],[useruuid],[alertdeptuuid],[alertuseruuid],[createduseruuid],[updateduseruuid],[deptid],[customeremployeeid1],[customerdeptid1],[writetype] from fxt_web_zhongqingtengyuzhiye.dbo.pro_customer_follow with(nolock) 具体错误信息为:com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:26) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.util.RdbmsException.asQueryException(RdbmsException.java:97) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Task.startRead(CommonRdbmsReader.java:220) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.sqlserverreader.SqlServerReader$Task.startRead(SqlServerReader.java:79) ~[sqlserverreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]
**显示为脏数据的这条数据重新导入时又成功了,数据完全没有变过,但是这个表经常报这种错,请问这种问题怎么解决,如何排查出错原因**
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the reported SQL Server logs and the reader entry points in CommonRdbmsReader.java, SqlServerReader.java, and ReaderRunner.java. Reproduce the intermittent “The connection is closed” failure and trace how the connection error becomes dirty data and stops the task. Done means the failure cause is identified and the issue defines a tested handling or recovery change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100