并发时 Communication 统计有异常
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
在项目实践中同时触发两个job任务 datax 打印出日志任务
2021-07-14 19:56:22.531 INFO 14772 --- [job-26] com.alibaba.datax.core.job.JobContainer :
任务启动时刻 : 2021-07-14 19:56:12
任务结束时刻 : 2021-07-14 19:56:22
任务总计耗时 : 10s
任务平均流量 : 2.49KB/s
记录写入速度 : 17rec/s
读出记录总数 : 170
读写失败总数 : 0
: Total 170 records, 25495 bytes | Speed 2.49KB/s, 17 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2021-07-14 19:56:22.763 INFO 14772 --- [job-852] com.alibaba.datax.core.job.JobContainer :
任务启动时刻 : 2021-07-14 19:56:12
任务结束时刻 : 2021-07-14 19:56:22
任务总计耗时 : 10s
任务平均流量 : 2.49KB/s
记录写入速度 : 17rec/s
读出记录总数 : 170
读写失败总数 : 0
其中第一个任务真实交换量是170条数据 第二个任务的交换量不是170条数据。
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at com.alibaba.datax.core.job.JobContainer and inspect how Communication statistics are collected or reported when two jobs run concurrently. Reproduce the reported two-job scenario and compare each job's record totals; done means each job's log reports its own exchange count rather than sharing another job's value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100