Transient failure of KafkaSupervisorTest.testCheckpointForInactiveTaskGroup
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
```
Tests run: 64, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 24.667 sec <<< FAILURE! - in org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest
testCheckpointForInactiveTaskGroup[numThreads = 1](org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest) Time elapsed: 0.157 sec <<< FAILURE!
java.lang.AssertionError:
java.lang.AssertionError:
Unexpected method call TaskRunner.getRunningTasks():
at org.easymock.internal.MockInvocationHandler.invoke(MockInvocationHandler.java:44)
at org.easymock.internal.ObjectMethodsFilter.invoke(ObjectMethodsFilter.java:94)
at com.sun.proxy.$Proxy44.getRunningTasks(Unknown Source)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor$1.getTaskLocation(KafkaSupervisor.java:312)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor.checkpointTaskGroup(KafkaSupervisor.java:1531)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor.checkTaskDuration(KafkaSupervisor.java:1461)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor.runInternal(KafkaSupervisor.java:898)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor$RunNotice.handle(KafkaSupervisor.java:611)
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisor$2.run(KafkaSupervisor.java:385)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
expected null, but was:
at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testCheckpointForInactiveTaskGroup(KafkaSupervisorTest.java:2126)
```
The problem is that the supervisor periodically executes runNotice and this can potentially call every method of the supervisor which in turn requires to mock everything using EasyMock.
I think it's better to refactor the whole KafkaSupervisorTest to be based on a sort of more controllable mockup environment by creating taskRunner, taskMaster, taskStorage, and taskClient for test purpose instead of making them using EasyMock.
Contributor guide
Research direction
Start with KafkaSupervisorTest.testCheckpointForInactiveTaskGroup at KafkaSupervisorTest.java:2126 and trace the periodic runNotice path through KafkaSupervisor.runInternal, checkTaskDuration, and checkpointTaskGroup. Review how taskRunner, taskMaster, taskStorage, and taskClient are mocked, then rerun the test repeatedly; done means the test uses a controllable test environment and no longer fails from unexpected periodic calls.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100