[Improvement] Remove SparkConsoleProgressBar in favor of SparkProgressMonitor
- Dominant language
- Scala
- Stars
- 2.4k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
Now in spark engine, we have two implementations of progress,
In #2119, ported the implementation of Spark ConsoleProgressBar, using `KyuubiStatement#getQueryLog` to pull logs.
In #2493, ported some implementations of Hive on Spark, using `GetOperationStatus`.
The functionality is duplicated, we can remove the implementation of `SparkConsoleProgressBar`.
### How should we improve?
https://github.com/apache/incubator-kyuubi/pull/2923#issuecomment-1161337819
- Introduce configuration to decide whether progress is output to `TProgressUpdateResp` or written to operation log, which is compatible with more hive client versions.
- If a query generates many stages, using `SparkProgressMonitor` will output the progress of each stage to the operation log, and the log will be larger.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Contributor guide
Assessment
This issue has not been assessed yet.