[FEATURE] Introduce a Spark catalog to generate TeraSort data
- Dominant language
- Scala
- Stars
- 2.4k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the feature
https://github.com/ehiggs/spark-terasort
### Motivation
The TeraSort is widely used for testing, we can implement a catalog-based data generate to simply the data preparation work.
### Describe the solution
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes. I can submit a PR independently to improve.
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
- [ ] No. I cannot submit a PR at this time.
Contributor guide
Research direction
Start by reviewing the linked spark-terasort project and Kyuubi's catalog architecture to identify the entry points for a catalog-based data generator. Define how the catalog should expose TeraSort generation and validate that it simplifies data preparation with appropriate tests or examples.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100