alibaba / alibaba/DataX

定时任务 datax 数据同步出现 datax执行完毕,数据还没有完全写入目标表

Open
#1,686 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

我自己在项目里面集成了datax,然后通过定时任务,在同一时刻,比如都是上午11点去执行2个定时任务,一个拉取班级数据,一个拉取学生数据,都是在同一个数据源去拉取,就会出现问题,比如数据实际并没有完全写到目标表,datax日志显示已经执行完了, 过几分钟数据又完全写到目标表了。但是单个的定时任务,比如11点只拉取学生数据,反正不是两个线程同时操作datax, 就不会出现这个问题,datax执行完毕,数据完全就写进去目标表了

Contributor guide

No contributing guide indexed for this repository

Research direction

No files or tests are named. Reproduce the case with two scheduled DataX jobs reading the same source at the same time, compare the completion logs with the target-table state, and determine what conditions make writes appear after execution has finished.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.