Bug: SparkClient treats retry states as terminal failures

### What happened?

### Summary

`SparkClient.wait_for_job_status()` can fail too early when a `SparkApplication` is being retried by Spark Operator.

The SDK currently maps several non-terminal Spark Operator states to `SparkJobStatus.FAILED`. As a result, callers may receive a failure before the operator has completed its configured retry flow.

### Current behavior

`SparkJobStatus.from_operator_state()` maps these operator states to `FAILED`:

- `SUBMISSION_FAILED`
- `FAILING`
- `PENDING_RERUN`
- `INVALIDATING`
- `UNKNOWN`

`wait_for_job_status()` raises as soon as it sees `SparkJobStatus.FAILED`.

For example, with a retry-enabled `RestartPolicy`, a job may validly transition through:

```text
SUBMISSION_FAILED
→ PENDING_RERUN
→ SUBMITTED
→ RUNNING
→ COMPLETED
```

Today, the SDK can raise during `SUBMISSION_FAILED` or `PENDING_RERUN`, preventing the caller from waiting for the retry to complete.

### Proposed solution

- Add SDK status values for non-terminal states, such as `RETRYING`, `SUSPENDED`, and `UNKNOWN`.
- Map retry/transition states such as `PENDING_RERUN`, `INVALIDATING`, and retryable submission failures to a non-terminal SDK status.
- Preserve the raw Spark Operator state and `errorMessage` on `SparkJob` so callers can diagnose the current lifecycle state.
- Update `wait_for_job_status()` to raise only for a terminal operator `FAILED` state.
- Treat previously unknown operator states as `UNKNOWN` rather than immediately converting them to `FAILED`.

### Acceptance criteria

- [ ] Every Spark Operator `ApplicationStateType` has a mapping test.
- [ ] A job transitioning from `PENDING_RERUN` to `RUNNING` and then `COMPLETED` does not raise from `wait_for_job_status()`.
- [ ] A terminal `FAILED` job still raises when `FAILED` is not one of the requested target states.
- [ ] `SparkJob` exposes the raw operator state and error message when available.
- [ ] Existing public status values remain backward compatible.
- [ ] Documentation explains how retrying and terminal job states are represented.

### References

- Spark Operator lifecycle design: https://kubeflow.github.io/spark-operator/docs/design.html
- Spark Operator `ApplicationStateType`: https://github.com/kubeflow/spark-operator/blob/master/api/v1beta2/sparkapplication_types.go
- Related LFX SparkClient observability/status work: https://github.com/kubeflow/sdk/issues/655

### What did you expect to happen?

SparkClient should distinguish terminal failures from retry and transition states.

Spark Operator documents `COMPLETED` and `FAILED` as terminal states. Other states may still transition or be retried depending on the configured restart policy.

`wait_for_job_status()` should continue polling while a job is retrying or transitioning, and raise only when the operator reaches a true terminal failure.

### Environment

Kubernetes version:
```bash
$ kubectl version
Client Version: v1.31.0

```
Kubeflow Trainer version:
```bash
$ kubectl get pods -n kubeflow -l app.kubernetes.io/name=trainer -o jsonpath="{.items[*].spec.containers[*].image}"

```
Kubeflow Python SDK version:
```bash
$ pip show kubeflow

```

### Impacted by this bug?

Give it a 👍 We prioritize the issues with most 👍