facebookresearch / facebookresearch/CodeGen

Question for validation and test sets

Open
#58 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
777
Forks
144
PR merge metrics
No merged PRs in 30d

Description

Hi,

as the paper "Unsupervised Translation of programming languages" mentioned, there are 852 parallel functions. So I checked the data in this repo [fold](https://github.com/facebookresearch/CodeGen/tree/main/data/transcoder_evaluation_gfg) and found (each file contain one function code with unit test cases, actually there are 852 union filenames in python/java/cpp folder):
- 698 cpp functions
- 717 java functions
- 702 python functions

the number of the test/validate dataset sizes is different as mentioned in Table 4 of the raw paper :
- c++ 466/231
- java 234/481
- python 237/463

And another question is function pairs number in Table 5 of the raw paper. I'm wander why the C++ -> java hava 481 tests functions while the java -> c++ only hava 466 test functions. If my understanding is right, there should have a same number of tests for giving parallel functions (java to c++ or c++ to java). Why the count of test functions if different for c++->java and java->c++ ? (same for other language pairs)

really thanks

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.