huggingface / huggingface/datasets

Cannot create a dataset with relative audio path

Open
#7,313 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
22k
Forks
3.4k
Avg merge
5d 7h
Merged PRs (30d)
17

Description

### Describe the bug

Hello! I want to create a dataset of parquet files, with audios stored as separate .mp3 files. However, it says "No such file or directory" (see the reproducing code).

### Steps to reproduce the bug

Creating a dataset
```
from pathlib import Path
from datasets import Dataset, load_dataset, Audio

Path('my_dataset/audio').mkdir(parents=True, exist_ok=True)
Path('my_dataset/audio/file.mp3').touch(exist_ok=True)
Dataset.from_list(
[{'audio': {'path': 'audio/file.mp3'}}]
).to_parquet('my_dataset/data.parquet')
```

Result:
```
# my_dataset
# ├── audio
# │ └── file.mp3
# └── data.parquet
```

Trying to load the dataset
```
dataset = (
load_dataset('my_dataset', split='train')
.cast_column('audio', Audio(sampling_rate=16_000))
)
dataset[0]

>>> FileNotFoundError: [Errno 2] No such file or directory: 'audio/file.mp3'
```

### Expected behavior

I expect the dataset to load correctly.

I've found 2 workarounds, but they are not very good:
1. I can specify an absolute path to the audio, however, when I move the folder or upload to HF it will stop working.
2. I can set `'path': 'file.mp3'`, and load with `load_dataset('my_dataset', data_dir='audio')` - it seems to work, but does this mean that anyone from Hugging Face who wants to use this dataset should also pass the `data_dir` argument, otherwise it won't work?

### Environment info

datasets 3.1.0, Ubuntu 24.04.1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.