[CORE] Incorrect wholestage id in WholeStageTransformerExec
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Backend
VL (Velox)
### Bug description
In vanilla spark, the stage id of the `WholeStageCodegenExec` always starts with 1 in each SQL query
However in Gluten, the stage id is generated from a global counter, which accumulates across different queries. This number can grow very large in a long running spark session.
### Gluten version
_No response_
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Research direction
Start by tracing WholeStageTransformerExec and comparing its stage-id handling with WholeStageCodegenExec, the two execution classes named in the report. Reproduce the behavior across multiple SQL queries and identify how the counter is scoped; done means stage IDs follow the per-query behavior described and regression coverage verifies it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100