JuliaML / JuliaML/MLDatasets.jl

Better Train, test, val split for graphs

Open
#158 3 comments 0 reactions 0 assignees View on GitHub
gsoc
Dominant language
Julia
Stars
239
Forks
51
PR merge metrics
No merged PRs in 30d

Description

## Problems

1. Splitting graphs is a bit more complicated than normal data. Graphs can be split based on node, edge or whole graph itself. We should be able to support all of that with minimal alterations. Currently, graph splits in MLDatases.jl have no specification.
2. The API for accessing train, test and val split is inconsistent.

## Proposed solution
I propose the following API:
1. Splits should be present in the metadata. We can access splits using `splits = dataset.metadata.split`
2. The split is a named tuple for train, test and val. Is empty if no predefined split is present.
3. Each split is a named tuple/struct which contains type of split $\in$ [node, edge, graph] and the `value` of the split.
4. There can be two or more splits depending on the feature you are splitting on. There are such examples in OGBDatasets. So it makes sense to make the `value` of the split a dict or named tuple.

This should ideally handle most of the cases. We only support non-dynamic graphs for now, I have not worked with dynamic graphs. Most likely we can support splits with little to no changes.

This recursive structure of splits can be confusing for the end-user. We can support APIs in MLUtils.jl like:

```julia
train_data, test_data, val_data = split(dataset)
# If no split is present in the dataset, all data will training data or error out.

train_data = split(dataset, :train)

# If we want to choose edge split
train_data = split(dataset, :train, :edge)

# Get training split for a feature
train_data = split(dataset, :train, :feature_name)
```
Keeping this a discussion for now.

cc: @CarloLucibello

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.