Cached dataset is not used when explicitly passing the cache_dir parameter

### Describe the bug

Hi, we are pre-downloading a dataset using snapshot_download(). When loading this exact dataset with load_dataset() the cached snapshot is not used. In both calls, I provide the cache_dir parameter.

### Steps to reproduce the bug

```
from datasets import load_dataset, concatenate_datasets
from huggingface_hub import snapshot_download

def download_ds(name: str):
snapshot_download(repo_id=name, repo_type="dataset", cache_dir="G:/Datasets/cache")

def prepare_ds():
audio_ds = load_dataset("openslr/librispeech_asr", num_proc=4, cache_dir="G:/Datasets/cache")
print(sfw_ds.features)

if __name__ == '__main__':
download_ds("openslr/librispeech_asr")
prepare_ds()
```

### Expected behavior

I'd expect that the cached version of the dataset is used. Instead, the same dataset is downloaded again to the default cache directory.

### Environment info

Windows 11
datasets==4.0.0
Python 3.12.11