facebookresearch / facebookresearch/CodeGen
Question for validation and test sets
- Dominant language
- Python
- Stars
- 777
- Forks
- 144
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
as the paper "Unsupervised Translation of programming languages" mentioned, there are 852 parallel functions. So I checked the data in this repo [fold](https://github.com/facebookresearch/CodeGen/tree/main/data/transcoder_evaluation_gfg) and found (each file contain one function code with unit test cases, actually there are 852 union filenames in python/java/cpp folder):
- 698 cpp functions
- 717 java functions
- 702 python functions
the number of the test/validate dataset sizes is different as mentioned in Table 4 of the raw paper :
- c++ 466/231
- java 234/481
- python 237/463
And another question is function pairs number in Table 5 of the raw paper. I'm wander why the C++ -> java hava 481 tests functions while the java -> c++ only hava 466 test functions. If my understanding is right, there should have a same number of tests for giving parallel functions (java to c++ or c++ to java). Why the count of test functions if different for c++->java and java->c++ ? (same for other language pairs)
really thanks
Contributor guide
Assessment
This issue has not been assessed yet.