google-deepmind / google-deepmind/deepmind-research

PolyGen Question: Train-Test Split

Open
#288 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
15.2k
Forks
2.9k
PR merge metrics
No merged PRs in 30d

Description

Hello,

I'm working with 3D shapes with Transformers and currently looking closer at the PolyGen paper.

In [Colab](https://colab.research.google.com/github/deepmind/deepmind-research/blob/master/polygen/sample-pretrained.ipynb) a pre-trained model is given.
I would like to get some quantitative metrics on the generated quality of the meshes, e.g. [MMD/COV scores](https://arxiv.org/abs/1812.02822), but for this task I need to know exactly which samples of ShapeNet were used for training and which for testing. The idea is to sample a generated set G and compare it to the test set A.

The information given in the paper is sadly not enought.
> "We train all our models on the ShapeNet Core V2 dataset (Chang et al., 2015), which we subdivide into 92.5% training, 2.5% validation and 5% testing splits." - PolyGen, 3.1. Training Details

@charlienash
Is the train-test split following any rules, which would allow to reconstruct the test set or are they just randomly sampled?

Sincerely Gregor

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.