huggingface / huggingface/course

A misprint in the "Big data? 🤗 Datasets to the rescue!" chapter of the NLP Course?

Open
#767 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
4.2k
Forks
1.4k
Avg merge
13m
Merged PRs (30d)
1

Description

There is the following code in the ["The magic of memory mapping"](https://huggingface.co/learn/nlp-course/chapter5/4#the-magic-of-memory-mapping) section:
```
print(f"Number of files in dataset : {pubmed_dataset.dataset_size}")
size_gb = pubmed_dataset.dataset_size / (1024**3)
print(f"Dataset size (cache file) : {size_gb:.2f} GB")
```
It seems there should be `"Number of bytes in dataset"` instead of `"Number of files in dataset"`, since the number of rows is 15 518 009 and dividing `pubmed_dataset.dataset_size` by `1024**3` suggests measuring information rather than the number of files.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.