apache / apache/gluten

[VL][CI] Reduce redundancy in Spark 3.5 SMJ test workflows

Open
#11,997 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Description

Currently, there are two Spark 3.5 test jobs dedicated for testing SMJ. The entire Spark 3.5 tests are run but most of them are not related to SMJ logic. This creates unnecessary redundancy. We may consider to pick up a few related test suites for validation.

### Gluten version

None

Contributor guide

Open the contributing guide

Research direction

Start by locating the two Spark 3.5 SMJ test jobs and compare the suites they execute. Identify the suites relevant to SMJ validation, then update the workflow so it avoids redundant full Spark 3.5 runs and verify the revised jobs in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
ci-cd, testing-qa
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.