mlcommons / mlcommons/mobile_app_open
Use the full dataset for accuracy validation in the integration test
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 71
- Forks
- 31
- Avg merge
- 6d 10h
- Merged PRs (30d)
- 2
Description
In our integration test we have a check to validate the accuracy results of the benchmarks. Currently, the app uses a tiny subset of the full dataset, so its result may not be reliable.
We should use the full dataset instead. Since this test is run exclusively on our CI, the full dataset can be stored privately on Google Cloud.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the integration test that validates benchmark accuracy and trace how it currently selects the tiny dataset. Review how CI can access a private full dataset on Google Cloud; done means the integration test runs against the full dataset and validates the resulting accuracy reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, gcp
- Domain
- data, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100