apache / apache/uniffle

Flaky Test: ShuffleTaskManagerTest#removeShuffleDataWithHdfsTest

Open
#612 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
454
Forks
172
Avg merge
5d 17h
Merged PRs (30d)
5

Description

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.

### Describe the flaky test

```
Error: Tests run: 14, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 85.858 s <<< FAILURE! - in org.apache.uniffle.server.ShuffleTaskManagerTest
Error: removeShuffleDataWithHdfsTest Time elapsed: 15.048 s <<< ERROR!
java.lang.RuntimeException: Shuffle data commit timeout for 10000 ms
at org.apache.uniffle.server.ShuffleTaskManager.commitShuffle(ShuffleTaskManager.java:259)
at org.apache.uniffle.server.ShuffleTaskManagerTest.removeShuffleDataWithHdfsTest(ShuffleTaskManagerTest.java:427)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
```

### Actions URL

https://github.com/apache/incubator-uniffle/actions/runs/4191298244/jobs/7265939805

### Parent issue

#1733

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by running ShuffleTaskManagerTest#removeShuffleDataWithHdfsTest and inspect the failure reported at ShuffleTaskManagerTest.java:427. Read ShuffleTaskManager.java around commitShuffle at line 259, then compare the linked Actions run for the timeout. Done means the test completes reliably without the shuffle data commit timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.