Results summarizer fails when results folder is empty
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43
- Forks
- 60
- Avg merge
- 19h 56m
- Merged PRs (30d)
- 3
Description
This error occurs:
Traceback (most recent call last):
File "/opt/python3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/python3.9/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/mlperf_logging/result_summarizer/__main__.py", line 3, in <module>
result_summarizer.main()
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/mlperf_logging/result_summarizer/result_summarizer.py", line 626, in main
_update_summaries(args.folder)
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/mlperf_logging/result_summarizer/result_summarizer.py", line 597, in _update_summaries
strong_scaling_summary, weak_scaling_summary = summarize_results(
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/mlperf_logging/result_summarizer/result_summarizer.py", line 517, in summarize_results
strong_scaling_scores = _compute_strong_scaling_scores(
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/mlperf_logging/result_summarizer/result_summarizer.py", line 330, in _compute_strong_scaling_scores
benchmark_scores[benchmark] = _compute_olympic_average(
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/mlperf_logging/result_summarizer/result_summarizer.py", line 185, in _compute_olympic_average
return sum_of_scores * 1.0 / len(countable_scores)
ZeroDivisionError: float division by zero
A suggested solution would be to add a previous check in package checker to avoid this "ugly" error
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with mlperf_logging/result_summarizer/result_summarizer.py, especially _compute_olympic_average, _compute_strong_scaling_scores, and _update_summaries; run the result summarizer against an empty results folder to reproduce the traceback. Done means the empty-folder case no longer raises the reported ZeroDivisionError and the existing summarization behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100