deepmodeling / deepmodeling/LAMBench

[Code scan] Pressure task reports success_rate with the wrong denominator

Open Beginner friendly
#434 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
20
Forks
11
Avg merge
30m
Merged PRs (30d)
4

Description

This issue was found by a Codex global repository scan of tracked non-test files at commit `8c93925cb10b401b2b83c738bd9263fd74474468`.

### Relevant code
https://github.com/deepmodeling/LAMBench/blob/8c93925cb10b401b2b83c738bd9263fd74474468/lambench/tasks/calculator/pressure/pressure.py#L57-L84

### Impact
`num_samples` is incremented only after a successful calculation, while `num_fails` is incremented for failed attempts. The final formula is `(num_samples - num_fails) / num_samples`, which can underreport success and can crash when every sample fails.

For example, with one successful sample and one failed sample, the current formula reports `(1 - 1) / 1 = 0.0` instead of the expected `1 / 2 = 0.5`.

### Suggested fix
Track total attempts separately from successful samples. Compute `success_rate = num_successes / total_attempts`, and return controlled `NaN` or `None` metrics plus `0.0` success when there are no successful samples.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in lambench/tasks/calculator/pressure/pressure.py at lines 57-84 and trace how successful samples, failed attempts, and the final metric are counted. Exercise the pressure task with mixed and all-failing samples, then update the calculation so success uses total attempts and no-success cases return controlled metrics with a 0.0 success rate.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.