huggingface / huggingface/datasets

config paths that start with ./ are not valid as hf:// accessed repos, but are valid when accessed locally

Open
#7,727 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
22k
Forks
3.4k
Avg merge
5d 7h
Merged PRs (30d)
17

Description

### Describe the bug

```
- config_name: some_config
data_files:
- split: train
path:
- images/xyz/*.jpg
```
will correctly download but

```
- config_name: some_config
data_files:
- split: train
path:
- ./images/xyz/*.jpg
```
will error with `FileNotFoundError` due to improper url joining. `load_dataset` on the same directory locally works fine.

### Steps to reproduce the bug

1. create a README.md with the front matter of the form

```
- config_name: some_config
data_files:
- split: train
path:
- ./images/xyz/*.jpg
```

2. `touch ./images/xyz/1.jpg`
3. Observe this directory loads with `load_dataset("filesystem_path", "some_config")` correctly.
4. Observe exceptions when you load this with `load_dataset("repoid/filesystem_path", "some_config")`

### Expected behavior

`./` prefix should be interpreted correctly

### Environment info

datasets 4.0.0
datasets 3.4.0

reproduce

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.