Proper link prediction evaluation in DistDGL
- Dominant language
- Python
- Stars
- 14.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
DistDGL does not allow two graphs for inference. In link prediction this will create problem of either not correct evaluation if testing edges are included in the graph or a lot of custom code to do it properly. I have some ideas I can share with the team.
**Main concern**
Imagine you train a GNN and the testing edges exists in the graph. Naturally, you will see the performance in the training set improving as the model trains. You will also see the performance in the testing edge improve and at a certain epoch K it will start degrading. The problem is that the overfitting will happen much earlier than epoch K, but since the testing edges exist in the graph we will not be able to see it. This will lead to a huge problem in production where the inference happens usually in unobserved edges.
The main solution here is to support a testing and a training graph in the DistDGL as we do in DGL.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.