deepmodeling / deepmodeling/deepmd-kit

tests: decouple API consistency coverage from numerical consistency coverage

Open
#5,373 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
2k
Forks
649
Avg merge
6d 18h
Merged PRs (30d)
15

Description

## Summary
API-consistency checks and numerical-consistency checks currently appear to share similar parameter matrices in some suites.

These two categories have very different cost/benefit profiles:
- API checks validate metadata / interface behavior
- numerical checks validate actual outputs / serialization / backend agreement

They should not necessarily use the same coverage strategy.

## Motivation
When API checks inherit the same large parameter matrix as numerical consistency tests, CI pays a large cost for combinations that may add little incremental confidence.

This creates unnecessary duplication in heavy suites.

## Proposed direction
Decouple API-consistency coverage from numerical-consistency coverage.

For example:
- keep API checks on a much smaller representative matrix
- reserve broader matrices for numerical agreement / serialization checks where they matter more
- define separate shared helpers for API-vs-numerical coverage levels if needed

## Scope
Apply across consistency suites, not just descriptors.

## Acceptance criteria
- API consistency tests use smaller representative matrices than numerical consistency tests
- test structure makes the distinction between API and numerical coverage explicit
- overall CI cost is reduced without weakening the intended API guarantees

---
Authored by OpenClaw (model: gpt-5.4)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.