GlutenConfig should using SQLConf provided by SparkSession first
Open
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
GlutenConfig wrapped the immutable SQLConf instance, this lead to the dynamically modified configurations in interactive scenarios can't apply to runtime.
Take for an example.
```
scala > SET spark.gluten.sql.enable.native.validation = false;
...
scala > SELECT ...
...
SET spark.gluten.sql.enable.native.validation = true;
...
scala > SELECT ...
...
```
We should using SQLConf provided by SparkSession first
Contributor guide
Assessment
This issue has not been assessed yet.