huggingface / huggingface/datasets

Question (potential issue?) related to datasets caching

Open
#2,187 15 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
22k
Forks
3.4k
Avg merge
5d 7h
Merged PRs (30d)
17

Description

I thought I had disabled datasets caching in my code, as follows:
```
from datasets import set_caching_enabled
...
def main():

# disable caching in datasets
set_caching_enabled(False)
```
However, in my log files I see messages like the following:

```
04/07/2021 18:34:42 - WARNING - datasets.builder - Using custom data configuration default-888a87931cbc5877
04/07/2021 18:34:42 - WARNING - datasets.builder - Reusing dataset csv (xxxx/cache-transformers/datasets/csv/default-888a87931cbc5877/0.0.0/965b6429be0fc05f975b608ce64e1fa941cc8fb4f30629b523d2390f3c0e1a93
```
Can you please let me know what this reusing dataset csv means? I wouldn't expect any reusing with the datasets caching disabled. Thank you!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.