[VL][CI] Reduce redundancy in Spark 3.5 SMJ test workflows
Open
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
Currently, there are two Spark 3.5 test jobs dedicated for testing SMJ. The entire Spark 3.5 tests are run but most of them are not related to SMJ logic. This creates unnecessary redundancy. We may consider to pick up a few related test suites for validation.
### Gluten version
None
Contributor guide
Research direction
Start by locating the two Spark 3.5 SMJ test jobs and compare the suites they execute. Identify the suites relevant to SMJ validation, then update the workflow so it avoids redundant full Spark 3.5 runs and verify the revised jobs in CI.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- ci-cd, testing-qa
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100