mlcommons / mlcommons/algorithmic-efficiency
Add dataset setup tests
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 425
- Forks
- 78
- PR merge metrics
- No merged PRs in 30d
Description
Description
Most of the code in data_setup.py is untested.
There are a few challenges for these tests:
- datasets are very large (total just under 2TB total I believe)
- some of them require manual steps (getting the links after signing the user agreements etc, I don't think we can check in the urls).
We can at a minimum test the datasets that are downloaded via tfds (ogbg and wmt) and add some unit tests for the other datasets.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading data_setup.py and identifying the tfds-backed ogbg and wmt dataset paths, along with setup logic for datasets requiring manual links. Add tests for the accessible download paths and unit tests for the remaining dataset setup behavior without requiring the large datasets or unavailable URLs. Done means the relevant setup code has meaningful automated test coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100