Race condition with SLURM restart

Nobody has claimed this yet.

## 🐛 Bug

This might have a really obvious solution that I haven't found, but how is one supposed to handle two simultaneous SLURM jobs with automatic restart? I submit them both from the same directory. After a (long) time, `job 0` begins and at some point later the `job 1` begins. `job 0` is eventually interrupted and saves a checkpoint `hpc_ckpt_0`, and subsequently `job 1` is interrupted and saves `hpc_ckpt_1`. Now they're both requeued. But of course `job 0` runs first and will load the latest checkpoint `hpc_ckpt_1`.

Is there some more sophisticated solution to this other than just running from different directories? Is there anywhere in the `CheckpointConnector` that would make sense to handle this case, or is it too niche?

## Expected Behaviour

If two jobs are interrupted, they should have some way of associating the checkpoint to the job. Possibly by submitting the name of the checkpoint in the requeue of the job. Maybe this makes more sense as a feature request?

## Version

I'm running PyLightning v1.5.10.

cc @borda @awaelchli

Start by reading CheckpointConnector and the SLURM restart and checkpoint handling around it. Reproduce the two-job, shared-directory scenario described in the issue, then determine how a checkpoint can remain associated with its job; done means requeued jobs no longer load one another’s checkpoints.

Written by the indexing model from the issue text.