HfHubHTTPError: 429 Client Error: Too Many Requests for URL when trying to access Fineweb-10BT on 4A100 GPUs using SLURM

### Describe the bug

I am trying to run some finetunings on 4 A100 GPUs using SLURM using axolotl training framework which in turn uses Huggingface's Trainer and Accelerate on [Fineweb-10BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb), but I end up running into 429 Client Error: Too Many Requests for URL error when I call next(dataloader_iter). Funny is, that I can run some test fine tuning (for just 200 training steps) in 1 A100 GPU using SLURM. Is there any rate limiter set for querying dataset? I could run the fine tuning with the same settings (4 A100 GPUs in SLURM) last month.

### Steps to reproduce the bug

You would need a server installed with SLURM

1. Create conda environment
1.1 conda create -n example_env -c conda-forge gxx=11 python=3.10
1.2 conda activate example_env
1.3 pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
1.4 conda install nvidia/label/cuda-12.4.0::cuda-toolkit
1.5 Download flash_attn-2.7.4.post1+cu12torch2.5cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
1.6 pip3 install packaging
1.7 pip3 install ninja
1.8 pip3 install mlflow
1.9 Clone https://github.com/calvintanama/axolotl.git
1.10 `cd` to `axolotl`
1.11 pip3 install -e '.[deepspeed]'

2. Run the training
2.1. Create a folder called `config_run` in axolotl directory
2.2. Copy `config/phi3_pruned_extra_pretrain_22_29_bottleneck_residual_8_a100_4.yaml` to `config_run`
2.3. Change yaml file in the `config_run` accordingly
2.4. Change directory and conda environment name in `jobs/train_phi3_22_29_bottleneck_residual_8_a100_4_temp.sh`
2.5. `jobs/train_phi3_22_29_bottleneck_residual_8_a100_4_temp.sh`

### Expected behavior

This should not cause any error, but gotten

```
File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/accelerate/data_loader.py", line 552, in __iter__
[rank3]: current_batch = next(dataloader_iter)
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 701, in __next__
[rank3]: data = self._next_data()
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 757, in _next_data
[rank3]: data = self._dataset_fetcher.fetch(index) # may raise StopIteration
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 33, in fetch
[rank3]: data.append(next(self.dataset_iter))
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/accelerate/data_loader.py", line 338, in __iter__
[rank3]: for element in self.dataset:
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 2266, in __iter__
[rank3]: for key, example in ex_iterable:
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 1866, in __iter__
[rank3]: for key, example in self.ex_iterable:
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 1084, in __iter__
[rank3]: yield from self._iter()
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 1263, in _iter
[rank3]: for key, transformed_example in outputs:
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 1258, in
[rank3]: outputs = (
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 1244, in iter_outputs
[rank3]: for i, key_example in inputs_iterator:
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 1106, in iter_batched_inputs
[rank3]: for key, example in iterator:
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 1866, in __iter__
[rank3]: for key, example in self.ex_iterable:
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 1535, in __iter__
[rank3]: for x in self.ex_iterable:
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/iterable_dataset.py", line 374, in __iter__
[rank3]: for key, pa_table in self.generate_tables_fn(**gen_kwags):
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/packaged_modules/parquet/parquet.py", line 90, in _generate_tables
[rank3]: if parquet_fragment.row_groups:
[rank3]: File "pyarrow/_dataset_parquet.pyx", line 386, in pyarrow._dataset_parquet.ParquetFileFragment.row_groups.__get__
[rank3]: File "pyarrow/_dataset_parquet.pyx", line 393, in pyarrow._dataset_parquet.ParquetFileFragment.metadata.__get__
[rank3]: File "pyarrow/_dataset_parquet.pyx", line 382, in pyarrow._dataset_parquet.ParquetFileFragment.ensure_complete_metadata
[rank3]: File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/datasets/utils/file_utils.py", line 827, in read_with_retries
[rank3]: out = read(*args, **kwargs)
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/huggingface_hub/hf_file_system.py", line 1013, in read
[rank3]: return super().read(length)
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/fsspec/spec.py", line 1941, in read
[rank3]: out = self.cache._fetch(self.loc, self.loc + length)
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/fsspec/caching.py", line 234, in _fetch
[rank3]: self.cache = self.fetcher(start, end) # new block replaces old
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/huggingface_hub/hf_file_system.py", line 976, in _fetch_range
[rank3]: hf_raise_for_status(r)
[rank3]: File "/home/hk-project-test-p0023745/cd7437/miniconda3/envs/llmpruning_train_temp/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 482, in hf_raise_for_status
[rank3]: raise _format(HfHubHTTPError, str(e), response) from e
[rank3]: huggingface_hub.errors.HfHubHTTPError: 429 Client Error: Too Many Requests for url: https://huggingface.co/datasets/HuggingFaceFW/fineweb/resolve/0f039043b23fe1d4eed300b504aa4b4a68f1c7ba/sample/10BT/006_00000.parquet
```

### Environment info

- datasets 3.5.0
- torch 2.5.1
- transformers 4.46.2