theislab / theislab/interscale
Mask all nodes once during loading
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12
- Forks
- 1
- Avg merge
- 12h 51m
- Merged PRs (30d)
- 1
Description
Description of feature
Given a batch of $i$ PyG objects with all different number of nodes $N_i$.
The current _spatial_node_loader randomly masks $L$ nodes, such as $L = P * min(N)$ where $P \in [0,1]$ is the percentage of nodes to be masked and $min(N)$ are the number of nodes of the smallest graph.
Problem: This approach potentially misses nodes from underrepresented categories (e.i. rare cell types) due to two reasons: 1) random selection of nodes doesn't consider the imbalanced dataset, 2) only masking a small fraction of nodes each time, especially considering that graph sizes might vary a lot.
An example dataset is provided in tests/test_geome_dataloader.py
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the _spatial_node_loader implementation and the example dataset in tests/test_geome_dataloader.py. Run the relevant dataloader tests to understand the current random masking of L = P * min(N) nodes across graphs. Done means loading a batch masks nodes once while addressing the stated risk of missing underrepresented categories.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100