作业同步ORACLE数据时,经常出现读取不到数据,或者无法往目标库写入数据的情况
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
作业同步ORACLE数据时,经常出现读取不到数据,或者无法往目标库写入数据的情况,没有任何报错存在,一直就是以下这种状态
日志内容摘取:
[INFO] 2022-09-16 10:07:56.713 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:07:56.214 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:08:16.715 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:08:16.216 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:08:26.715 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:08:26.216 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:08:36.716 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:08:36.217 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:08:56.718 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:08:56.218 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:09:06.720 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:09:06.219 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:09:26.721 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:09:26.220 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:09:36.722 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:09:36.221 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:09:56.724 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:09:56.222 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2022-09-16 10:09:56.223 [job-0] INFO VMInfo -
[delta cpu info] =>
curDeltaCpu | averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00% | -1.00%
[delta memory info] =>
NAME | used_size | used_percent | max_used_size | max_percent
PS Eden Space | 30.72MB | 12.00% | 30.72MB | 12.00%
Code Cache | 4.74MB | 98.44% | 4.74MB | 98.44%
Compressed Class Space | 2.43MB | 92.47% | 2.43MB | 92.47%
PS Survivor Space | 0.00MB | 0.00% | 0.00MB | 0.00%
PS Old Gen | 10.47MB | 1.53% | 10.47MB | 1.53%
Metaspace | 21.51MB | 98.32% | 21.51MB | 98.32%
[delta gc info] =>
NAME | curDeltaGCCount | totalGCCount | maxDeltaGCCount | minDeltaGCCount | curDeltaGCTime | totalGCTime | maxDeltaGCTime | minDeltaGCTime
PS MarkSweep | 1 | 1 | 1 | 1 | 0.019s | 0.019s | 0.019s | 0.019s
PS Scavenge | 1 | 1 | 1 | 1 | 0.009s | 0.009s | 0.009s | 0.009s
[INFO] 2022-09-16 10:10:06.725 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:10:06.224 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:10:26.727 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:10:26.225 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:10:46.729 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:10:46.226 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:10:56.729 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:10:56.226 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:11:06.730 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:11:06.227 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:11:26.732 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:11:26.228 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:11:46.734 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:11:46.229 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:12:06.736 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:12:06.230 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
[INFO] 2022-09-16 10:12:26.737 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[66] - -> 2022-09-16 10:12:26.231 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the DataxTask log entry and reproduce the reported Oracle synchronization job using the same configuration and logs. Compare reader and writer behavior when the job remains at zero records with no error; done means the job reports failures clearly and successfully reads from and writes to the target database.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100