alibaba / alibaba/DataX

作业同步ORACLE数据时,经常出现读取不到数据,或者无法往目标库写入数据的情况

Open
#1,520 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

作业同步ORACLE数据时,经常出现读取不到数据,或者无法往目标库写入数据的情况,没有任何报错存在,一直就是以下这种状态

日志内容摘取:
[INFO] 2022-09-16 10:07:56.713 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:07:56.214 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:08:16.715 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:08:16.216 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:08:26.715 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:08:26.216 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:08:36.716 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:08:36.217 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:08:56.718 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:08:56.218 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:09:06.720 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:09:06.219 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:09:26.721 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:09:26.220 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:09:36.722 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:09:36.221 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:09:56.724 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:09:56.222 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2022-09-16 10:09:56.223 [job-0] INFO VMInfo -
[delta cpu info] =>
curDeltaCpu | averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00% | -1.00%


[delta memory info] =>
NAME | used_size | used_percent | max_used_size | max_percent
PS Eden Space | 30.72MB | 12.00% | 30.72MB | 12.00%
Code Cache | 4.74MB | 98.44% | 4.74MB | 98.44%
Compressed Class Space | 2.43MB | 92.47% | 2.43MB | 92.47%
PS Survivor Space | 0.00MB | 0.00% | 0.00MB | 0.00%
PS Old Gen | 10.47MB | 1.53% | 10.47MB | 1.53%
Metaspace | 21.51MB | 98.32% | 21.51MB | 98.32%

[delta gc info] =>
NAME | curDeltaGCCount | totalGCCount | maxDeltaGCCount | minDeltaGCCount | curDeltaGCTime | totalGCTime | maxDeltaGCTime | minDeltaGCTime
PS MarkSweep | 1 | 1 | 1 | 1 | 0.019s | 0.019s | 0.019s | 0.019s
PS Scavenge | 1 | 1 | 1 | 1 | 0.009s | 0.009s | 0.009s | 0.009s

[INFO] 2022-09-16 10:10:06.725 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:10:06.224 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:10:26.727 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:10:26.225 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:10:46.729 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:10:46.226 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:10:56.729 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:10:56.226 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:11:06.730 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:11:06.227 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:11:26.732 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:11:26.228 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:11:46.734 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:11:46.229 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:12:06.736 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:12:06.230 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:12:26.737 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:12:26.231 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the DataxTask log entry and reproduce the reported Oracle synchronization job using the same configuration and logs. Compare reader and writer behavior when the job remains at zero records with no error; done means the job reports failures clearly and successfully reads from and writes to the target database.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.