hf_transfer download sometimes hangs without failing or succeeding

### System Info

hf_transfer = 0.1.6
huggingface_hub = 0.21.4

### Reproduction

We have not been able to reproduce and it is hard to really give details, since it happens very rarely in a system that deletes the downloaded file after using it, so no real way to know the state it leaves the machine in. But the behaviour is that we will start a download and it just will never return, the callback is not called either, nothing.

Eventually our system closes the process and we do not have visibility into it other than the logs it produces. The lib produces none logs, only some logs from our code:

```
Downloading https://xxx.r2.cloudflarestorage.com/yyy to /tmp/file
```

and there it hangs

### Expected behavior

It should either fail or succeed, but not hang

*****

Thanks for this amazing project, by the way. The speed is just amazing! I wish I was able to give more hints, but it really shows us nothing.