huggingface / huggingface/datasets

[📄 Docs] Create a `datasets` performance guide.

Open
#3,829 1 comment 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
22k
Forks
3.4k
Avg merge
5d 7h
Merged PRs (30d)
17

Description

## Brief Overview
Downloading, saving, and preprocessing large datasets from the `datasets` library can often result in [performance bottlenecks](https://github.com/huggingface/datasets/issues/3735). These performance snags can be challenging to identify and to debug, especially for users who are less experienced with building deep learning experiments.

## Feature Request
Could we create a performance guide for using `datasets`, similar to:
* [Better performance with the `tf.data` API](https://github.com/huggingface/datasets/issues/3735)
* [Analyze `tf.data` performance with the TF Profiler](https://www.tensorflow.org/guide/data_performance_analysis)

This performance guide should detail practical options for improving performance with `datasets`, and enumerate any common best practices. It should also show how to use tools like the PyTorch Profiler or the TF Profiler to identify any performance bottlenecks (example below).

![image](https://user-images.githubusercontent.com/3712347/156859152-a3cb9565-3ec6-4d39-8e77-56d0a75a4954.png)

## Related Issues
* [wiki_dpr pre-processing performance #1670](https://github.com/huggingface/datasets/issues/1670)
* [Adjusting chunk size for streaming datasets #3499](https://github.com/huggingface/datasets/issues/3499)
* [how large datasets are handled under the hood #1004](https://github.com/huggingface/datasets/issues/1004)
* [using map on loaded Tokenizer 10x - 100x slower than default Tokenizer? #1830](https://github.com/huggingface/datasets/issues/1830)
* [Best way to batch a large dataset? #315](https://github.com/huggingface/datasets/issues/315)
* [Saving processed dataset running infinitely #1911](https://github.com/huggingface/datasets/issues/1911)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.