Automate supported Spark versions and publish a Spark image matrix

### What feature you would like to be added?

Define a supported Apache Spark version policy, automatically propose updates when new supported Spark patch releases become available, and publish a documented matrix of Spark runtime images for each supported Spark version and architecture.

This issue supersedes the stale-closed #1982 by adding automated version discovery, lifecycle management, verification, and matrix publication rather than relying on manual image builds.

### Why is this needed?

Spark Operator releases should not force users onto one Spark version or require every user to maintain custom Spark runtime images. Manual version tracking causes supported images and documentation to drift behind Apache Spark releases. Users also need a clear compatibility matrix showing which Spark Operator releases are tested with which Spark versions and which image tags are available.

### Describe the solution you would like

- Establish a support policy, for example latest patch releases from the currently supported Spark minor lines.
- Keep supported Spark versions in one machine-readable source of truth.
- Add scheduled automation that detects new Apache Spark releases and opens reviewed pull requests updating that source, build inputs, tests, and generated documentation.
- Build and publish Spark runtime images for every supported Spark version across `linux/amd64` and `linux/arm64`.
- Use unambiguous immutable tags containing Spark version and, where needed, Hadoop/Java variants.
- Publish a generated compatibility matrix covering Spark Operator version, Spark version, image tag, architecture, Java/Hadoop variant, and support status.
- Exercise each supported image in CI with representative SparkApplication tests before publication.
- Define deprecation and removal behavior for Spark versions leaving the support window.
- Generate provenance, checksums, or signatures according to Kubeflow release standards.

### Describe alternatives you have considered

- Publish only one Spark image per Spark Operator release. This couples unrelated release cycles and forces upgrades.
- Require users to build all Spark images themselves. This creates duplicated work and inconsistent security maintenance.
- Maintain multiple images manually. This does not prevent version or documentation drift.

### Additional context

Related stale issue: #1982

The repository already contains `spark-docker/` and e2e support for loading a Spark image, providing a foundation for matrix builds and validation.

This work should coordinate with release automation tracked by #3063.

### Love this feature?

Give it a 👍 We prioritize the features with most 👍

Start by reading the existing spark-docker/ directory and e2e support for loading a Spark image, then review the release automation tracked by #3063. The work is done when supported versions are discovered and updated automatically, images are validated across the stated architectures, and generated documentation publishes the compatibility matrix with support and variant details.

Written by the indexing model from the issue text.