[Bug] there is no such file or directory
- Dominant language
- Scala
- Stars
- 2.4k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
Pre-Description:
1、run sql on spark engine
2、spark engine write and read from hive
3、create hive table as orc or parquet
when i execute sql like “insert overwrite table tpch.ods_tpch_lineitem_d_1 select distinct * from tpch.ods_tpch_lineitem_d;”
i check the hdfs path,find that the kyuubi spark engine will delete the hdfs path and will create the path at the last dag stage

but,there is a issue, if the sql run failed and do not reach the last stage,then the hdfs path of hive table will lost. when i close the spark session and open a new session, run sql “insert overwrite table tpch.ods_tpch_lineitem_d_1 select distinct * from tpch.ods_tpch_lineitem_d;” it will can not find the hdfs path and the task will failed.
### Affects Version(s)
1.7.3/1.8.0
### Kyuubi Server Log Output
_No response_
### Kyuubi Engine Log Output
_No response_
### Kyuubi Server Configurations
_No response_
### Kyuubi Engine Configurations
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [ ] No. I cannot submit a PR at this time.
Contributor guide
Research direction
No source files, tests, or logs are named. Start by reproducing the shown SQL against the Spark engine with Hive-backed ORC or Parquet tables and trace when the HDFS path is removed; done means a failed insert does not leave the table path unavailable to a subsequent session.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, scala, spark, sql
- Domain
- data-engineering, databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100