[Improvement] Add a integration test case for Spark with MiniYarnCluster
Open
good first issue
- Dominant language
- Java
- Stars
- 454
- Forks
- 172
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 5
Description
We use Spark local mode to write integration test, but we can't cover some cases, such as https://github.com/Tencent/Firestorm/pull/187, so we should add a integration test case for Spark with MiniYarnCluster
Contributor guide
Research direction
Start by locating the existing Spark local-mode integration tests and read the limitations described in PR 187. Then investigate how MiniYarnCluster is used in this repository and define the test setup needed to exercise the missing cases; done means the Spark integration suite covers cases that local mode cannot reproduce.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java, spark
- Domain
- distributed-systems, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100