deepmodeling / deepmodeling/LAMBench

[Code scan] Missing or failed inference-efficiency results can crash or change schema

Open Beginner friendly
#439 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
20
Forks
11
Avg merge
30m
Merged PRs (30d)
4

Description

This issue was found by a Codex global repository scan of tracked non-test files at commit `8c93925cb10b401b2b83c738bd9263fd74474468`.

### Relevant code
https://github.com/deepmodeling/LAMBench/blob/8c93925cb10b401b2b83c738bd9263fd74474468/lambench/metrics/vishelper/results_fetcher.py#L108-L119
https://github.com/deepmodeling/LAMBench/blob/8c93925cb10b401b2b83c738bd9263fd74474468/lambench/metrics/utils.py#L129-L150

### Impact
`fetch_inference_efficiency_results_for_one_model()` logs a warning when the database query returns zero or multiple records, but it still indexes `task_results[0]`. A missing record therefore raises `IndexError` during result generation.

There is also a schema inconsistency: the successful aggregation branch returns `standard_deviation`, while the failure branch returns `std_time`. Consumers of the generated JSON see different keys depending on whether any system failed.

### Suggested fix
Return `None` or a consistent failure object when `len(task_results) != 1`, before indexing. Also make the failure branch return the same keys as the success branch, for example `average_time`, `standard_deviation`, and `success_rate`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in lambench/metrics/vishelper/results_fetcher.py around lines 108-119 and trace the result handling into lambench/metrics/utils.py around lines 129-150. Check the zero-, multiple-, success-, and failure-result paths. Done means missing or ambiguous records no longer crash, and generated JSON uses the same result keys in both success and failure cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
2/5
Estimated time
Half a day
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.