huggingface / huggingface/hf_transfer
Why is downloading extremely slow at the end of each shard?
Open
- Dominant language
- Rust
- Stars
- 580
- Forks
- 41
- Avg merge
- 7m
- Merged PRs (30d)
- 1
Description
```bash
export HF_HUB_ENABLE_HF_TRANSFER=1
huggingface-cli download Qwen/QwQ-32B-Preview
```
```
model-00007-of-00017.safetensors: 23%|█████████████████ | 912M/3.90G [00:23<00:45, 65.8MB/s]
model-00007-of-00017.safetensors: 99%|████████████████████████████████████████████████████████████████████████▌| 3.88G/3.90G [05:45<01:32, 228kB/s
```
This only happens in some clusters.
Could anyone help me figure out the possible cause?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.