meta-pytorch / meta-pytorch/data
online hard-example mining under Multi-GPU (DDP)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
🚀 The feature
Do you think it would be possible to have an off the shelf component like the one described at https://github.com/Lightning-AI/pytorch-lightning/discussions/1170?
Motivation, pitch
Sometime on unbalanced dataset it could be interesting to make specific sampling related to perplexity or othermetrics and loss values.
It could be also interesting for implementing some type of online curriculum learning/sampling solution.
Alternatives
No response
Additional context
Write and maintain a custom component.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked PyTorch Lightning discussion and review this repository's data-loading utilities to identify an appropriate extension point. Clarify how sampling from perplexity or loss should work with Multi-GPU DDP, and define completion around an agreed component scope and distributed-training behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- data, distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100