deepmodeling / deepmodeling/LAMBench
[Code scan] Pressure task reports success_rate with the wrong denominator
- Dominant language
- Python
- Stars
- 20
- Forks
- 11
- Avg merge
- 30m
- Merged PRs (30d)
- 4
Description
This issue was found by a Codex global repository scan of tracked non-test files at commit `8c93925cb10b401b2b83c738bd9263fd74474468`.
### Relevant code
https://github.com/deepmodeling/LAMBench/blob/8c93925cb10b401b2b83c738bd9263fd74474468/lambench/tasks/calculator/pressure/pressure.py#L57-L84
### Impact
`num_samples` is incremented only after a successful calculation, while `num_fails` is incremented for failed attempts. The final formula is `(num_samples - num_fails) / num_samples`, which can underreport success and can crash when every sample fails.
For example, with one successful sample and one failed sample, the current formula reports `(1 - 1) / 1 = 0.0` instead of the expected `1 / 2 = 0.5`.
### Suggested fix
Track total attempts separately from successful samples. Compute `success_rate = num_successes / total_attempts`, and return controlled `NaN` or `None` metrics plus `0.0` success when there are no successful samples.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in lambench/tasks/calculator/pressure/pressure.py at lines 57-84 and trace how successful samples, failed attempts, and the final metric are counted. Exercise the pressure task with mixed and all-failing samples, then update the calculation so success uses total attempts and no-success cases return controlled metrics with a 0.0 success rate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 74/100