[Feature] SparkGenericCatalog can only use the hive warehouse dir, and cannot use the paimon warehouse dir to isolate it from the hive warehouse dir
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Motivation
When creating Hive Catalog in Paimon, Flink can specify the Paimon-specific Warehouse Dir, but the Hive Catalog created by SparkCatalog in SparkGenericCatalog cannot specify the Paimon-specific Warehourse Dir. The functions of the two should be consistent to provide better user experience.
### Solution
Add the spark.sql.paimon.warehourse.dir configuration option to support SparkGenericCatalog configuration Paimon Warehouese Dir
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at SparkGenericCatalog and trace how its Hive Catalog warehouse directory is configured. Check the existing Flink-specific warehouse behavior and identify where the spark.sql.paimon.warehourse.dir option should be read. Done means SparkGenericCatalog can use a Paimon-specific warehouse directory independently of the Hive warehouse directory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100