mlcommons / mlcommons/inference
Refactor TEST01 message so it's less confusing
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 650
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 6
Description
Right now if the part II of TEST01 fails, it will show up as "TEST FAIL" in the log which is confusing for the submitter.
We should refactor the message such that the user knows there's a fallback that will be run if the current test fails, also why the part II fails and part III should be run.
@pgmpablo157321 FYI
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Search the repository for TEST01 and the existing "TEST FAIL" log message to locate the relevant test flow. Review how parts II and III and the fallback are handled, then update the message so the fallback, failure reason, and next step are clear and verify the affected benchmark test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Refactor
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100