huggingface / huggingface/datasets
Feature request: being able to disable chmod for flat permission filesystems
- Dominant language
- Python
- Stars
- 22k
- Forks
- 3.4k
- Avg merge
- 5d 7h
- Merged PRs (30d)
- 17
Description
### Feature request
For flat permission file systems, `os.chmod` simply fails when trying to cache datasets - in some places such as: https://github.com/huggingface/datasets/blob/main/src/datasets/arrow_dataset.py#L4043-L4048 for arrow datasets.
This seems to be optional to obtained desired behavior as simply commenting it out makes our workload work as expected. Hence the feature request would be to see if it would be possible to expose an API to disable `os.chmod` operations when users asks to do so (e.g. through an env variable?)
### Motivation
Being able to save properly datasets on flat permission filesystems
### Your contribution
Can test the feature if implemented
Contributor guide
Assessment
This issue has not been assessed yet.