Feature request: per-process Spot → on-demand fallback for Google Batch

New feature

Let a google-batch process run on Spot for its first N attempts and then automatically fall back to on-demand (STANDARD) for later attempts, controlled per process.

This is the gap between the two things that exist today: the global all-or-nothing google.batch.spot, and the per-process static scheduling.provisioningModel hint being added in #3530 / #7343. A static spot choice is a reliability downgrade for time-sensitive pipelines - a reclaim or a capacity stockout simply fails. This proposal is the attempt-aware fallback (spot with an automatic on-demand safety net.

Use case

Deploying to Google Batch with the majority of jobs on Spot for cost, but needing them to still complete reliably (and within a predictable turnaround). Two failure modes make static Spot hard to adopt:

1. Mid-run reclaim - a Spot VM is reclaimed while a task runs. google.batch.maxSpotAttempts retries, but only on Spot again - it never escalates, so under sustained Spot pressure the task keeps failing.
2. Can't obtain a Spot VM at submit time - during regional capacity pressure a Spot request may not be schedulable at all. Static spot waits/fails; static standard gives up the savings entirely.

A per-process "Spot for N attempts, then on-demand" policy captures Spot savings on the common path while guaranteeing the task still completes on on-demand when Spot is unavailable - making Spot safe to adopt broadly, including for latency-sensitive workloads.

Prior art / parity: AWS Batch already provides Spot → on-demand fallback natively in nf-amazon, via ordered compute environments in a job queue. Google Batch has no queue abstraction (a job gets exactly one provisioningModel), so the equivalent has to be expressed in the Nextflow layer. This brings the google-batch executor to parity - i.e. fallback isn't a platform-only capability.

Suggested implementation

A companion hints key in the same scheduling.* namespace as #7343, resolved in GoogleBatchTaskHandler alongside the static scheduling.provisioningModel:

process EXAMPLE {
    errorStrategy 'retry'
    maxRetries 3
    hints 'google-batch/scheduling.spotAttempts': '2'   // Spot for attempts 1–2, on-demand from attempt 3
}

• scheduling.spotAttempts = positive integer N. Attempts 1..N request Spot; attempts after N request on-demand (STANDARD).
• Resolve the effective ProvisioningModel per attempt in the task handler (the same place #7343 resolves scheduling.provisioningModel); when the hint is unset, behaviour is unchanged.
• The attempt count would be the greater of task.attempt and task.submitAttempt, so the fallback triggers for both failure modes: a mid-run reclaim (task.attempt, via errorStrategy 'retry') and a failure to obtain a Spot VM (task.submitAttempt, in combination with maxSubmitAwait).
• Composes with #7343's static scheduling.provisioningModel; distinct from the in-job maxSpotAttempts retry (which stays on Spot).

Happy to open a PR

Related: #3530, #7343, #7034

Start in GoogleBatchTaskHandler, where the issue says scheduling.provisioningModel is resolved, and compare the related behavior in #7343 and #3530. Trace task.attempt, task.submitAttempt, maxSubmitAwait, and maxSpotAttempts to determine how the effective model should change per attempt. Done means an optional scheduling.spotAttempts hint uses Spot for the first N attempts and STANDARD thereafter without changing unset behavior.