apache / apache/druid

Transient failure of KafkaSupervisorTest.testCheckpointForInactiveTaskGroup

Open
#6,296 2 comments 0 reactions 0 assignees View on GitHub
Area - Testing Bug Flaky test stale
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

```
Tests run: 64, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 24.667 sec <<< FAILURE! - in org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest
testCheckpointForInactiveTaskGroup[numThreads = 1](org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest) Time elapsed: 0.157 sec <<< FAILURE!
java.lang.AssertionError:
java.lang.AssertionError:
Unexpected method call TaskRunner.getRunningTasks():
at org.easymock.internal.MockInvocationHandler.invoke(MockInvocationHandler.java:44)
at org.easymock.internal.ObjectMethodsFilter.invoke(ObjectMethodsFilter.java:94)
at com.sun.proxy.$Proxy44.getRunningTasks(Unknown Source)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor$1.getTaskLocation(KafkaSupervisor.java:312)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor.checkpointTaskGroup(KafkaSupervisor.java:1531)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor.checkTaskDuration(KafkaSupervisor.java:1461)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor.runInternal(KafkaSupervisor.java:898)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor$RunNotice.handle(KafkaSupervisor.java:611)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor$2.run(KafkaSupervisor.java:385)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
expected null, but was:
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testCheckpointForInactiveTaskGroup(KafkaSupervisorTest.java:2126)
```

The problem is that the supervisor periodically executes runNotice and this can potentially call every method of the supervisor which in turn requires to mock everything using EasyMock.

I think it's better to refactor the whole KafkaSupervisorTest to be based on a sort of more controllable mockup environment by creating taskRunner, taskMaster, taskStorage, and taskClient for test purpose instead of making them using EasyMock.

Contributor guide

Open the contributing guide

Research direction

Start with KafkaSupervisorTest.testCheckpointForInactiveTaskGroup at KafkaSupervisorTest.java:2126 and trace the periodic runNotice path through KafkaSupervisor.runInternal, checkTaskDuration, and checkpointTaskGroup. Review how taskRunner, taskMaster, taskStorage, and taskClient are mocked, then rerun the test repeatedly; done means the test uses a controllable test environment and no longer fails from unexpected periodic calls.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.