deepmodeling / deepmodeling/LAMBench

[Code scan] NEB zero failure rates are emitted as missing metrics

Open
#435 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
20
Forks
11
Avg merge
30m
Merged PRs (30d)
4

Description

This issue was found by a Codex global repository scan of tracked non-test files at commit `8c93925cb10b401b2b83c738bd9263fd74474468`.

### Relevant code
https://github.com/deepmodeling/LAMBench/blob/8c93925cb10b401b2b83c738bd9263fd74474468/lambench/tasks/calculator/neb/neb.py#L62-L86

### Impact
The NEB failure percentage is computed from only the rows where `error > ERROR_THRESHOLD`. If a reaction type has zero failures, that type is absent from the numerator groupby, so the division can produce a missing value instead of an explicit `0.0` failure rate.

A perfect category should be represented as `0.0`, not as a missing metric. Otherwise downstream JSON and leaderboard plots can treat a valid result as absent.

### Suggested fix
Build the denominator counts first, reindex the failure counts against those types with `fill_value=0`, then divide. Add a regression case where one NEB type has no failures.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.