[Bug] Spark Driver cannot be shutdown at the end of a task

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [x] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.

### Describe the bug

when i use k8s deploy spark task, the task driver pod is not compact.

### Affects Version(s)

1.9.3

### Kyuubi Server Log Output

```logtalk

```

### Kyuubi Engine Log Output

```logtalk
25/04/09 03:00:06 INFO SparkTBinaryFrontendService: Service[SparkTBinaryFrontend] is initialized.
25/04/09 03:00:06 INFO SparkTBinaryFrontendService: Service[SparkTBinaryFrontend] is started.
25/04/09 03:00:06 INFO SparkTBinaryFrontendService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
25/04/09 03:00:06 INFO SparkTBinaryFrontendService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
25/04/09 03:00:08 ERROR SparkTBinaryFrontendService: Error fetching results:
25/04/09 03:00:08 INFO SparkTBinaryFrontendService: Received request of closing SessionHandle [09f53cfd-2220-4e1e-8dd7-6b61dc8b2f52]
25/04/09 03:00:08 INFO SparkTBinaryFrontendService: Finished closing SessionHandle [09f53cfd-2220-4e1e-8dd7-6b61dc8b2f52]
25/04/09 03:00:10 INFO SparkTBinaryFrontendService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
25/04/09 03:00:11 INFO SparkTBinaryFrontendService: Received request of closing SessionHandle [0b434ecc-111a-44f5-b9d5-9ad29f67ff83]
25/04/09 03:00:11 INFO SparkTBinaryFrontendService: Finished closing SessionHandle [0b434ecc-111a-44f5-b9d5-9ad29f67ff83]
25/04/09 03:00:11 INFO SparkTBinaryFrontendService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
25/04/09 03:00:12 INFO SparkTBinaryFrontendService: Received request of closing SessionHandle [ab9e6ae9-0fef-4210-b432-286a9bf4885e]
25/04/09 03:00:12 INFO SparkTBinaryFrontendService: Finished closing SessionHandle [ab9e6ae9-0fef-4210-b432-286a9bf4885e]
25/04/09 03:00:12 INFO SparkTBinaryFrontendService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
25/04/09 03:00:13 INFO SparkTBinaryFrontendService: Received request of closing SessionHandle [84a5120e-4238-4a8b-9b4b-20720df1eabc]
25/04/09 03:00:13 INFO SparkTBinaryFrontendService: Finished closing SessionHandle [84a5120e-4238-4a8b-9b4b-20720df1eabc]
25/04/09 03:00:13 INFO SparkTBinaryFrontendService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
25/04/09 03:00:13 INFO SparkTBinaryFrontendService: Received request of closing SessionHandle [773ad9ba-687c-4004-a64a-0b0b7ef5089f]
25/04/09 03:00:13 INFO SparkTBinaryFrontendService: Finished closing SessionHandle [773ad9ba-687c-4004-a64a-0b0b7ef5089f]
25/04/09 03:00:13 INFO SparkTBinaryFrontendService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
25/04/09 03:04:16 INFO SparkSQLEngine: Service: [SparkTBinaryFrontend] is stopping.
25/04/09 03:04:16 INFO SparkTBinaryFrontendService: Service: [EngineServiceDiscovery] is stopping.
25/04/09 03:04:16 INFO SparkTBinaryFrontendService: Service[SparkTBinaryFrontend] is stopped.
25/04/09 03:04:16 INFO SparkTBinaryFrontendService: SparkTBinaryFrontend has stopped
25/04/09 03:08:19 INFO SparkTBinaryFrontendService: Received request of closing SessionHandle [7041cd34-ca41-4593-bbad-7979c9a071da]
25/04/09 03:08:19 INFO SparkTBinaryFrontendService: Finished closing SessionHandle [7041cd34-ca41-4593-bbad-7979c9a071da]
```

### Kyuubi Server Configurations

```yaml

```

### Kyuubi Engine Configurations

```yaml

```

### Additional context

_No response_

### Are you willing to submit PR?

- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [x] No. I cannot submit a PR at this time.

Start by reproducing the issue with Kyuubi 1.9.3, a Spark task deployed on Kubernetes, and the provided engine log sequence. Inspect the Spark engine shutdown and Kubernetes driver-pod lifecycle around session and task closure; done means the driver pod is removed after the task ends.

Written by the indexing model from the issue text.