huggingface / huggingface/transformers
denoising with sentence permutation, and language sampling
Open
Feature request
- Dominant language
- Python
- Stars
- 166k
- Forks
- 34.6k
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 276
Description
# 🚀 Feature request
## Motivation
When training or fine tuning models, data collator provided in huggingface isn't enough.
For example, if we want to further pretrain `mBART` or `XLM-R`, where language sampling or sentence permutation are needed, which is hard to do with huggingface datasets API since it loads all language datasets at first.
Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.