ephemeral disk migration can get stuck if upstream disk fails

### Nomad version
Nomad v1.7.6
BuildDate 2024-03-12T07:27:36Z
Revision 594fedbfbc4f0e532b65e8a69b28ff9403eb822e
### Operating system and Environment details
Debian 11

### Issue

I had a job fail to start after being rescheduled because it was blocked waiting for ephemeral disk migration. In this case, the node failed because its storage failed but the node itself remained reachable on the network and processes kept running (insofar as they didn't crash or fail on their own).

Nomad did correctly mark the node as failed and correctly rescheduled the job, but also tried to migrate the ephemeral disk. This operation only failed after several hours, when I rebooted the failed node.

This generated the following logging:

```
May 14 04:31:05 nomad01 nomad[1571988]: 2024-05-14T04:31:05.483+0200 [INFO] client.alloc_runner.task_runner: Task event: alloc_id=25557839-5ce5-4df3-5531-9ffc277d19f0 task=vector type=Received msg="Task received by client" failed=false
May 14 11:35:16 nomad01 nomad[1571988]: 2024-05-14T11:35:16.034+0200 [WARN] client.alloc_runner.runner_hook.migrate_disk: error migrating data from previous alloc: alloc_id=25557839-5ce5-4df3-5531-9ffc277d19f0 error="error getting snapshot from previous alloc \"d9f1c7be-f979-9cd0-075e-1bcace5f24a6\": Get \"http://192.168.1.103:4646/v1/client/allocation/d9f1c7be-f979-9cd0-075e-1bcace5f24a6/snapshot\": read tcp 192.168.0.17:45488->192.168.1.103:4646: read: connection reset by peer"
May 14 11:35:16 nomad01 nomad[1571988]: 2024-05-14T11:35:16.062+0200 [INFO] client.alloc_runner.task_runner: Task event: alloc_id=25557839-5ce5-4df3-5531-9ffc277d19f0 task=vector type="Task Setup" msg="Building Task Directory" failed=false
May 14 11:35:16 nomad01 nomad[1571988]: 2024-05-14T11:35:16.741+0200 [INFO] client.alloc_runner.task_runner: Task event: alloc_id=25557839-5ce5-4df3-5531-9ffc277d19f0 task=vector type=Started msg="Task started by client" failed=false

```

The node failed and the task was rescheduled at 04:31, and the migration blocked until the affected node was rebooted at 11:35 later that morning.

### Reproduction steps

Have a job with ephemeral disk migration and then kill the disk i/o on the node causing nomad to both remain reachable but also get marked as failed by nomad.

#### Expected Result

- Jobs that were rescheduled because the node running them failed should probably not attempt to migrate their ephemeral storage. I feel like it can be reasonably safely assumed to also have failed.
- Where migration _is_ attempted, reasonable network i/o timeouts should exist such that migration does not block without progress for more than a few minutes.

#### Actual Result

Job blocked for extended periods on data migration.