Migrate repo to modern machine learning benchmarking standards
Open
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 164
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
Specifically, making use of PyTorch Lightning, Wandb, and fixed seeds for reproducing results (some changes may be unrealistic due to constraints from already written paper)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the repository's current benchmarking and experiment setup, then compare it with the requested use of PyTorch Lightning, Wandb, and fixed seeds. Account for the constraints imposed by the existing paper. Done means the benchmarking workflow follows the agreed standards and produces reproducible results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100