Reduce Spark related CI tests running time
Open
from-jira
priority:critical
type:devtask
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
UT and FT totally took about 2 hours to finish, which seriously slows down the speed of developers, and limits the team to create unit tests in finer granularity. We should speed up these slow tests.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-7245
- Type: Task
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the Spark UT and FT suites responsible for the reported roughly two-hour runtime, then measure their current execution time. Done means the tests finish faster while preserving their existing behavior and coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- ci-cd, performance, testing
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100