facebookresearch / facebookresearch/CodeGen

Training script stuck (using eval only)

Open
#26 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
777
Forks
144
PR merge metrics
No merged PRs in 30d

Description

I'm trying to run
```
python codegen_sources/model/train.py --eval_only True --reload_model 'TransCoder_model_2.pth,TransCoder_model_2.pth' --data_path "test_dataset" --exp_name transcoder --dump_path 'dump' --lgs 'java_sa-python_sa' --bt_steps 'python_sa-java_sa-python_sa,java_sa-python_sa-java_sa' --ae_steps 'python_sa,java_sa' --mt_steps 'java_sa-python_sa,python_sa-java_sa' --encoder_only False --emb_dim 1024 --n_heads 8 --n_layers 0 --n_layers_encoder 6 --n_layers_decoder 6 --eval_bleu true --eval_computation true --has_sentences_ids true
```
but it is unable to find the following files in the test_dataset (downloaded from the transcoder doc)
```
test_dataset/train.java_sa.pth not found
test_dataset/valid.java_sa.pth not found
test_dataset/test.java_sa.pth not found
test_dataset/train.python_sa.pth not found
test_dataset/valid.python_sa.pth not found
test_dataset/test.python_sa.pth not found
test_dataset/train.java_sa-python_sa.java_sa.pth not found
test_dataset/train.java_sa-python_sa.python_sa.pth not found
```
and gets stuck after this log message:
```
SLURM job: False
0 - Number of nodes: 1
0 - Node ID : 0
0 - Local rank : 0
0 - Global rank : 0
0 - World size : 1
0 - GPUs per node : 1
0 - Master : True
0 - Multi-node : False
0 - Multi-GPU : False
0 - Hostname :
```

Any ideas why this might be happening?

I also tried running this translation script, which similarly seems to get stuck:
```
python -m codegen_sources.model.translate --src_lang python --tgt_lang java --model_path TransCoder_model_2.pth.1 --beam_size 1 < hello.py
adding to path /srv/home/akshit/CodeGen
INFO - 09/27/21 05:56:56 - 0:00:06 - ============ Model Reloading
INFO - 09/27/21 05:56:56 - 0:00:06 - Reloading encoder from TransCoder_model_2.pth.1 ...
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.