[Flaky Test] org.apache.uniffle.test.ReassignAndStageRetryTest
- Dominant language
- Java
- Stars
- 454
- Forks
- 172
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 5
Description
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [x] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### Describe the flaky test
```
Error: Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 26.563 s <<< FAILURE! - in org.apache.uniffle.test.ReassignAndStageRetryTest
Error: resultCompareTest Time elapsed: 17.907 s <<< ERROR!
org.apache.spark.SparkException:
Job aborted due to stage failure: Task 0 in stage 7.0 failed 1 times, most recent failure: Lost task 0.0 in stage 7.0 (TID 4) (fv-az1372-936.dr3av053rsgelju3xdt4uaoa0c.dx.internal.cloudapp.net executor driver): org.apache.uniffle.common.exception.RssException: Blocks read inconsistent: expected 1 blocks, actual 0 blocks
at org.apache.uniffle.common.util.RssUtils.checkProcessedBlockIds(RssUtils.java:418)
at org.apache.uniffle.client.impl.ShuffleReadClientImpl.checkProcessedBlockIds(ShuffleReadClientImpl.java:329)
at org.apache.spark.shuffle.reader.RssShuffleDataIterator.hasNext(RssShuffleDataIterator.java:132)
at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:31)
at org.apache.spark.shuffle.reader.RssShuffleReader$MultiPartitionIterator.hasNext(RssShuffleReader.java:324)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at org.apache.spark.shuffle.reader.RssFetchFailedIterator.hasNext(RssFetchFailedIterator.java:116)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage3.sort_addToSorter_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage3.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:759)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
```
### Actions URL
https://github.com/apache/uniffle/actions/runs/15130502479/job/42530441263
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Contributor guide
Research direction
Begin with org.apache.uniffle.test.ReassignAndStageRetryTest and reproduce resultCompareTest using the linked Actions run as context. Trace the expected-versus-actual block count through RssUtils.checkProcessedBlockIds, ShuffleReadClientImpl.checkProcessedBlockIds, and RssShuffleDataIterator.hasNext; done means identifying the intermittent cause and making the test pass reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- distributed-systems, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100