deepmodeling / deepmodeling/LAMBench
[Code scan] Missing or failed inference-efficiency results can crash or change schema
- Dominant language
- Python
- Stars
- 20
- Forks
- 11
- Avg merge
- 30m
- Merged PRs (30d)
- 4
Description
This issue was found by a Codex global repository scan of tracked non-test files at commit `8c93925cb10b401b2b83c738bd9263fd74474468`.
### Relevant code
https://github.com/deepmodeling/LAMBench/blob/8c93925cb10b401b2b83c738bd9263fd74474468/lambench/metrics/vishelper/results_fetcher.py#L108-L119
https://github.com/deepmodeling/LAMBench/blob/8c93925cb10b401b2b83c738bd9263fd74474468/lambench/metrics/utils.py#L129-L150
### Impact
`fetch_inference_efficiency_results_for_one_model()` logs a warning when the database query returns zero or multiple records, but it still indexes `task_results[0]`. A missing record therefore raises `IndexError` during result generation.
There is also a schema inconsistency: the successful aggregation branch returns `standard_deviation`, while the failure branch returns `std_time`. Consumers of the generated JSON see different keys depending on whether any system failed.
### Suggested fix
Return `None` or a consistent failure object when `len(task_results) != 1`, before indexing. Also make the failure branch return the same keys as the success branch, for example `average_time`, `standard_deviation`, and `success_rate`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in lambench/metrics/vishelper/results_fetcher.py around lines 108-119 and trace the result handling into lambench/metrics/utils.py around lines 129-150. Check the zero-, multiple-, success-, and failure-result paths. Done means missing or ambiguous records no longer crash, and generated JSON uses the same result keys in both success and failure cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- Half a day
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100