facebookresearch / facebookresearch/CodeGen
Outputted files have erroneous characters [bug on mac os]
- Dominant language
- Python
- Stars
- 777
- Forks
- 144
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I was trying to reproduce the results in the TransCoder paper, but ran into some issues when it computed the computational accuracy.
It seems that when writing things such as id’s or outputted programs to files or outputs in the log, the text has some issues.
For example, in ids.java_sa-python_sa.text.txt (created by create_reference_files in evaluator.py), the lines look like “CHECK_@@ WHE@@ THER_@@ GI@@ V@@ EN_@@ NUMBER_@@ EV@@ EN_@@ O@@ DD”.
The scripts outputted by the model (e.g. in eval_scripts/java_sa-python_sa.test/) that are used to compute the computational accuracy similarly have erroneous characters and spaces (which causes syntax errors), e.g.:
```
def f_filled is_@@ ap ( arr , n ) :
if n == 1 : return True
arr.sort ( )
d = arr [ 1 ] - arr [ 0 ]
for i in range ( 2 , n ) :
if arr [ i ] - arr [ i - 1 ] != d : return False
return True
```
If it is relevant, I am on Mac OS Catalina with python 3.9 and this is the command I have been running to evaluate the TransCoder models provided:
```
python codegen_sources/model/train.py \
--eval_only True \
--reload_model 'TransCoder_model_1.pth,TransCoder_model_2.pth' \
--data_path "test_dataset" \
--exp_name transcoder \
--dump_path 'dump' \
--lgs 'java_sa-python_sa' \
--bt_steps 'python_sa-java_sa-python_sa,java_sa-python_sa-java_sa' \
--ae_steps 'python_sa,java_sa' \
--mt_steps 'java_sa-python_sa,python_sa-java_sa' \
--encoder_only False \
--emb_dim 1024 \
--n_heads 8 \
--n_layers 0 \
--n_layers_encoder 6 \
--n_layers_decoder 6 \
--eval_bleu true \
--eval_computation true \
--has_sentences_ids true
```
Thank you.
Contributor guide
Assessment
This issue has not been assessed yet.