facebookresearch / facebookresearch/stopes
eng_twl_short_porn.txt missing (toxicity filtering NLLB)
Open
- Dominant language
- Python
- Stars
- 313
- Forks
- 47
- PR merge metrics
- No merged PRs in 30d
Description
Hi folks!
I noticed that your toxicity filter is using [eng_twl_short_porn.txt](https://github.com/facebookresearch/stopes/blob/main/stopes/pipelines/filtering/configs.py#L54) (it's hardcoded in the `ToxicityFilterConfig` in `stopes/stopes/pipelines/filtering/configs.py`) but I can't find that file anywhere?
Nor it is mentioned in the NLLB paper?
Am I missing something out here?
Thanks in advance!
Contributor guide
Assessment
This issue has not been assessed yet.