Adding delays between JDBC connection retries

Hello,

In my company we are building an internal platform that runs many ETL processes over Spark 4.1 with PySpark.
We are using an AWS RDS Postgres DB as well as custom Postgres deployments for usage with AGE etc., all read/write via Spark.
We have an issue with connection retries during RDS rollovers or other intermittent issues.
For writers this is not a problem as we can add our own delay with backoff and then retry the writer. However, For readers, this is a problem because at the point of materialization the reader is already baked into the pipeline and it does not have and delays or backoffs during retries and we cannot replace or recreate the reader once the the full ETL plan is already built by Spark.

I'm wondering if it would be possible to add configurations for delays and backoffs?
We need to be able to handle connection refusals of up to 20 seconds at a time (once every few days).

Thanks!

Start by tracing the Spark JDBC reader path at materialization, focusing on how connection refusals are retried, and compare it with the writer behavior described in the issue. Determine where configurable delay and backoff settings would apply, then add coverage for intermittent failures lasting up to 20 seconds; done means readers retry without rebuilding the ETL plan.

Written by the indexing model from the issue text.