apache / apache/uniffle

[Flaky Test] org.apache.uniffle.test.ReassignAndStageRetryTest

Open
#2,493 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
454
Forks
172
Avg merge
5d 17h
Merged PRs (30d)
5

Description

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [x] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.

### Describe the flaky test

```
Error: Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 26.563 s <<< FAILURE! - in org.apache.uniffle.test.ReassignAndStageRetryTest
Error: resultCompareTest Time elapsed: 17.907 s <<< ERROR!
org.apache.spark.SparkException:
Job aborted due to stage failure: Task 0 in stage 7.0 failed 1 times, most recent failure: Lost task 0.0 in stage 7.0 (TID 4) (fv-az1372-936.dr3av053rsgelju3xdt4uaoa0c.dx.internal.cloudapp.net executor driver): org.apache.uniffle.common.exception.RssException: Blocks read inconsistent: expected 1 blocks, actual 0 blocks
at org.apache.uniffle.common.util.RssUtils.checkProcessedBlockIds(RssUtils.java:418)
at org.apache.uniffle.client.impl.ShuffleReadClientImpl.checkProcessedBlockIds(ShuffleReadClientImpl.java:329)
at org.apache.spark.shuffle.reader.RssShuffleDataIterator.hasNext(RssShuffleDataIterator.java:132)
at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:31)
at org.apache.spark.shuffle.reader.RssShuffleReader$MultiPartitionIterator.hasNext(RssShuffleReader.java:324)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at org.apache.spark.shuffle.reader.RssFetchFailedIterator.hasNext(RssFetchFailedIterator.java:116)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage3.sort_addToSorter_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage3.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:759)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
```

### Actions URL

https://github.com/apache/uniffle/actions/runs/15130502479/job/42530441263

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Begin with org.apache.uniffle.test.ReassignAndStageRetryTest and reproduce resultCompareTest using the linked Actions run as context. Trace the expected-versus-actual block count through RssUtils.checkProcessedBlockIds, ShuffleReadClientImpl.checkProcessedBlockIds, and RssShuffleDataIterator.hasNext; done means identifying the intermittent cause and making the test pass reliably.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
distributed-systems, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.