✨ Add optional generic SPANK launch-time device validation

🤖 AI text below 🤖

Lead: @flowerthrower. Cross-repository coordination: @burgholzer.

Elevator pitch

Catch inaccessible devices before a task does expensive work without putting provider SDKs into slurmstepd. Keep this optional check separate from decisions about when Slurm should allocate resources.

Current state

The shared injection component is tracked separately. The existing Core open/status check supplies the agreed minimal semantics: open the selected device and accept IDLE or BUSY, not a submission/readiness guarantee.

Work and decisions

• Extend the shared source-only GPL SPANK component to invoke the MIT checker out of process.
• Establish correct hook timing from Slurm’s actual license/environment availability; do not assume selection is available at user-init.
• Use the effective job identity and configuration, bounded timeout, and safe child cleanup.
• Reject a managed task cleanly on failure without draining its node; test the specific hook return convention.
• Avoid repeated unbounded provider queries across large steps; prove any once-per-step caching against the real hook/process lifecycle.
• Keep unrelated jobs unaffected and do not attest licenses or authorize provider access.
• Include only when the safety tests pass; otherwise static migration proceeds without early validation and without legacy plugins.

Dependencies and release boundary

Parent: ✨ Centralize static-license QDMI deployment and optional SPANK injection. Requires ✨ Add shared source-only SPANK configuration injection and ✨ Add a reusable QDMI device access and status checker. 💡 Design QDMI availability-based Slurm scheduling is independent.

Acceptance and validation

• An isolated multi-task Slurm fixture covers success, unavailable/unknown device, authentication failure, timeout, missing/unrelated licenses, and checker failure.
• Failed validation prevents task launch without draining nodes or inheriting daemon credentials.
• No provider SDK/Core runtime is linked into SPANK; wheel/sdist packaging retains the intended MIT/GPL boundary.
• Target Core 4.1/MQSF with the shared setup and provider migrations. Include only if the safety tests pass; otherwise defer without blocking static migration.

Non-goals

Holding jobs pending before allocation; relying on QDMI 1.4 accepting-jobs metadata; retaining provider-specific validation implementations.

Shared validation fixture

Exercise the checker/plugin boundary in Core's common Dockerized Slurm fixture with provider configuration supplied by the integration tests. Reuse the existing open and IDLE/BUSY semantics; successful validation is not a promise of submission acceptance or prompt execution.