facebookresearch / facebookresearch/stopes
Prepare new data for NLLB-200
- Dominant language
- Python
- Stars
- 313
- Forks
- 47
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I'm trying to fine tune NLLB-200 model on new bilingual data. So I need to prepare my data using prepare_data pipeline: https://github.com/facebookresearch/stopes/tree/main/stopes/pipelines/prepare_data
there are my configs file:



My output directory is the following:

But I encountered a problem when fine tuning NLLb-200:
**File "/home/admin/khadija/fairseq/slurm_snapshot_code/2023-02-08T14_51_26.242208/fairseq/data/dictionary.py", line 238, in add_from_file
with open(PathManager.get_local_path(f), "r", encoding="utf-8") as fd:
FileNotFoundError: [Errno 2] No such file or directory: '/home/admin/khadija/prepare_data_output/data_bin/shard000/dict.ary_Arab.txt'
srun: error: slurmnode1: tasks 0-2: Exited with exit code 1**
Is Fairseq compatible with the new version of Stopes?
@Mortimerp9 @kauterry @gwenzek Can you help me please?
Contributor guide
Assessment
This issue has not been assessed yet.