alibaba / alibaba/DataX

TaskGroupContainer中task retry的npe,TaskGroupContainer line:146 左右

Open
#818 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

` TaskExecutor taskExecutor = removeTask(runTasks, taskId);
//上面从runTasks里移除了,因此对应在monitor里移除
taskMonitor.removeTask(taskId);
//失败,看task是否支持failover,重试次数未超过最大限制
if(taskCommunication.getState() == State.FAILED){
taskFailedExecutorMap.put(taskId, taskExecutor);
if(taskExecutor.supportFailOver() && taskExecutor.getAttemptCount() < taskMaxRetryTimes){
taskExecutor.shutdown(); //关闭老的executor
containerCommunicator.resetCommunication(taskId); //将task的状态重置
Configuration taskConfig = taskConfigMap.get(taskId);
taskQueue.add(taskConfig); //重新加入任务列表
}else{
failedOrKilled = true;
break;
}`

`//3.有任务未执行,且正在运行的任务数小于最大通道限制
Iterator iterator = taskQueue.iterator();
while (iterator.hasNext()) {
Configuration taskConfig = iterator.next();
Integer taskId = taskConfig.getInt(CoreConstant.TASK_ID);
int attemptCount = 1;
TaskExecutor lastExecutor = taskFailedExecutorMap.get(taskId);
if (lastExecutor != null) {
attemptCount = lastExecutor.getAttemptCount() + 1;
long now = System.currentTimeMillis();
//上次失败的task仍未结束
if (!lastExecutor.isShutdown()) {
lastExecutor.shutdown(); //再次尝试关闭
continue;
} else {
LOG.info("jobId[{}] taskId[{}] attemptCount[{}] has already shutdown",
this.jobId, taskId, lastExecutor.getAttemptCount());
}
}`

在code 1 从runTasks中取当前运行的task,然后当 taskExecutor未关闭,在code 2处runTasks这个未add,导致第二次循环进入,taskExecutor就为null

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in TaskGroupContainer around line 146 and trace the shown retry path from removeTask(runTasks, taskId) through taskFailedExecutorMap and taskQueue. Reproduce or trace the second loop when the failed TaskExecutor is not yet shut down; done means this path no longer produces a null TaskExecutor or NPE while preserving the retry handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.