huggingface / huggingface/datasets

DownloadConfig.proxies not work when load_dataset_builder calling HfApi.dataset_info

Open
#6,032 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
22k
Forks
3.4k
Avg merge
5d 7h
Merged PRs (30d)
17

Description

### Describe the bug

```python
download_config = DownloadConfig(proxies={'https': ''})
builder = load_dataset_builder(..., download_config=download_config)
```
But, when getting the dataset_info from HfApi, the http requests not using the proxies.

### Steps to reproduce the bug

1. Setup proxies in DownloadConfig.
2. Call `load_dataset_build` with download_config.
3. Inspect the call stack in HfApi.dataset_info.

![image](https://github.com/huggingface/datasets/assets/138426806/33e538a8-2e22-4e63-b634-343febe5324b)

### Expected behavior

DownloadConfig.proxies works for getting dataset_info.

### Environment info

https://github.com/huggingface/datasets/commit/406b2212263c0d33f267e35b917f410ff6b3bc00
Python 3.11.4

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.