deepmodeling / deepmodeling/deepmd-kit
docs: document features added since v3.1.3
- Dominant language
- Python
- Stars
- 2k
- Forks
- 649
- Avg merge
- 6d 18h
- Merged PRs (30d)
- 15
Description
## Summary
While reviewing `feat(...)` commits added since the last stable release (`v3.1.3`), I found a number of user-facing features that are not documented in `doc/` (or only appear as schema entries in the auto-generated argument reference from `deepmd/utils/argcheck.py`, with no explanatory prose, examples, or guide-page coverage).
This issue tracks the documentation gaps so they can be closed. A checklist of items to add is at the bottom.
## Fully undocumented features
### 1. TF2 (TensorFlow 2) eager backend + training workflow — `dp --tf2`
- Commits: `73de44b1f` (eager Array backend, SavedModel export), `26ad0fb83` (training/freeze/compress workflow, multi-task, finetune, TensorBoard)
- Implementation: `deepmd/backend/tf2.py:32` (`name = "TensorFlow2"`), the `deepmd/tf2/` package, `deepmd/tf2/entrypoints/`
- Gap: Not listed in `doc/backend.md` (only TF v1 / PyTorch / JAX / Paddle / DP), no tab in `doc/train/training.md`, `doc/freeze/freeze.md`, or `doc/freeze/compress.md`. The `DP_JIT` env var is documented (`doc/env.md:51`), but the backend itself is not.
### 2. `pt_expt` (PyTorch-Exportable) backend — `dp --pt_expt`
- Commits: `9bd1f16b1`, `2cbef365d`, `3f91293b8`, `23622361d`, etc.
- Implementation: `deepmd/backend/pt_expt.py:32` (`name = "PyTorch-Exportable"`), `deepmd/pt_expt/`
- Gap: Referenced in `doc/model/dpa2.md`, `doc/model/dpa4.md`, `doc/model/train-se-atten.md` via the `--lower-kind graph` / `.pt2` graph-native route, but **never introduced as a backend** in `doc/backend.md`. Its distinct capabilities (`.pt2`/AOTInductor export, LMDB datasets, multi-task training, `dp finetune`/`change-bias`/`compress` for pt_expt) have no coherent documentation. `doc/freeze/freeze.md` only shows TF/PT/Paddle.
### 3. JAX training workflow — `dp --jax train`
- Commit: `20874166a` (`feat(jax): add training`)
- Gap: The doc change `1de6de9c9` only *deleted* the line "this backend … has no support for training" from `doc/backend.md`; it added no positive statement. `doc/train/training.md` has TF/PyTorch/Paddle tabs but **no JAX tab**, and `dp --jax train` is not documented anywhere.
### 4. `dp test` stress output
- Commit: `330fa7558` (`feat: dp test stress`)
- Implementation: `deepmd/entrypoints/test.py:540` (`.s.out` detail file), `:825`–`:835`, `:902`, `:976` (Stress MAE/RMSE in eV/ų)
- Gap: `doc/test/test.md` has no mention of stress output files or stress metrics; `grep stress doc/test/` returns nothing.
### 5. `DeepEval.serialize` / `dp show serialization-tree`
- Commit: `e19bd188b` (`feat(show): add serialization-tree via DeepEval.serialize`)
- Implementation: `deepmd/main.py:1049` (adds the `"serialization-tree"` choice), `deepmd/entrypoints/show.py:153`
- Gap: `doc/model/show-model-info.md` lists `model-branch`, `type-map`, `descriptor`, `fitting-net`, `size`, `observed-type` — but **not** `serialization-tree`. No prose anywhere.
### 6. `dp freeze --hessian` (JAX backend)
- Commit: `44131151f` (`feat(jax): freeze models with hessian output`)
- Implementation: `deepmd/main.py:354` (`parser_frz` `--hessian` flag, "Supported backend: JAX")
- Gap: `doc/freeze/freeze.md` does not document `--hessian`. Worse, `doc/model/train-energy-hessian.md:94` and `doc/model/overall.md:61` still state Hessian models **cannot be frozen** — now incorrect for JAX.
### 7. Vesin neighbor list + pluggable NeighborList strategy (Python/ASE inference)
- Commit: `23622361d` (`feat(pt_expt): pluggable NeighborList strategy + O(N) vesin neighbor list for Python/ASE inference`)
- Implementation: `deepmd/pt_expt/utils/vesin_neighbor_list.py`, `deepmd/pt_expt/infer/deep_eval.py:190`/`:214`/`:280` (`nlist_backend` argument: `"auto"`/`"vesin"`/`"native"`)
- Gap: `doc/inference/python.md:61` ("External neighbor list algorithm") is TF-only and documents only ASE `NewPrimitiveNeighborList`; `vesin` appears nowhere in `doc/`.
## Thinly documented features (schema-only / passing mention)
These are registered in `deepmd/utils/argcheck.py`, so they appear in the auto-generated argument reference, but they have no explanatory prose, examples, or guide-page coverage.
### 8. Descriptor compression for dpmodel & JAX backends — `dp --dp compress`, `dp --jax compress`
- Commit: `f143171cf` (`feat(dpmodel): add descriptor compression`)
- Implementation: `deepmd/dpmodel/entrypoints/compress.py`, `deepmd/dpmodel/entrypoints/compress_common.py`
- Gap: `doc/freeze/compress.md:4` states "Supported backends: TensorFlow, PyTorch" and only shows `dp compress` / `dp --pt compress`. The new dpmodel/JAX routes are not mentioned.
### 9. Charge-spin conditioning (training-side config) — `add_chg_spin_ebd`, `default_chg_spin`, `dim_chg_spin`
- Commits: `e3fda4537` (C++ runtime), `f39a08128` (dpa3 decouple from fparam), `6984e4c63` (paddle)
- Implementation: `deepmd/utils/argcheck.py:694` (`doc_add_chg_spin_ebd`), `:1041`, `:2097`
- Gap: The LAMMPS *inference* keyword `charge_spin` is documented (`doc/third-party/lammps-command.md:66`), and DPA4 mentions `add_chg_spin_ebd` only for graph-eligibility (`doc/model/dpa4.md:506,603`). The **training config** is undocumented in `doc/model/dpa3.md` and `doc/model/train-energy-spin.md` (both have zero mentions of `chg_spin`/`add_chg_spin`).
### 10. `stat_file` / `stat_file_mode` training config (TensorFlow)
- Commit: `2b3bb05a7` (`feat(tf): support training stat_file`)
- Implementation: `deepmd/utils/argcheck.py:5300` (`doc_stat_file`), `:5348` (Argument), `deepmd/tf/model/stat_file.py`
- Gap: No prose in `doc/train/`; `grep stat_file doc/` returns nothing.
### 11. `so3_readout` DPA4/SeZM option (`"none"`/`"glu"`/`"mlp"`)
- Commit: `4b6506db4` (`feat(dpa4): so3_readout across pt + dpmodel/pt_expt backends`)
- Implementation: `deepmd/utils/argcheck.py:626` (`doc_so3_readout`), `:999` (Argument)
- Gap: Not in `doc/model/dpa4.md` prose (the page documents many other DPA4 options).
### 12. EMA (Exponential Moving Average) shadow model — `enable_ema`, `ema_decay`, `ema_ckpt_keep`
- Commit: `57f870ff4` (`feat(pt): add ema shadow model`)
- Implementation: `deepmd/utils/argcheck.py:5264` (`doc_enable_ema`), `:5269` (`doc_ema_decay`)
- Gap: No prose in `doc/train/`. The only doc mention is a passing reference to `ema_ckpt_keep` inside the `ckpt_keep_ratio` description (`doc/train/training-advanced.md:107`).
### 13. Force L2-norm / force-MAE loss option — `force_l2_norm`
- Commit: `3f52fa95a` (`feat(dp, pt): add force l2 norm loss & mae loss`)
- Implementation: `deepmd/utils/argcheck.py:4326` ("use L2 norm of force vectors for loss calculation …")
- Gap: Not in `doc/model/train-energy.md` loss prose.
### 14. Three-value Huber delta — `huber_delta`
- Commit: `14c349b42` (`feat(loss): support three-value huber delta`)
- Implementation: `deepmd/utils/argcheck.py:4482` (`huber_delta` Argument)
- Gap: `grep huber doc/` returns nothing.
### 15. Element type-wise bias in property fitting — `distinguish_types`
- Commit: `7cd0d191b` (`feat(property): Support element type-wise bias in property fitting`)
- Implementation: `deepmd/utils/argcheck.py:2827` (`doc_distinguish_types`), `:2872`
- Gap: Not in `doc/model/train-fitting-property.md`.
### 16. LMDB dataset support for training
- Commits: `565f4beec` (pt/dpmodel dataloader), `54f42d90c` (pt_expt), `9d6381602` (max/filter mode), `a3f548d7b` (full validation lmdb)
- Implementation: `deepmd/pt_expt/utils/` (LMDB dataset), referenced from pt/dpmodel dataloaders
- Gap: Not in `doc/data/` or `doc/train/` as a training data format. Mentioned only in `doc/dpa_adapt/input_formats.md:105` (as a dpa-adapt conversion target).
## Orphaned / navigation-broken
### 17. `doc/model/train-fitting-population.md` is missing from the toctree
- `doc/model/train-fitting-population.md` exists and is well written, but it is **not listed in `doc/model/index.rst`** toctree (which has `train-fitting-tensor`, `train-fitting-dos`, `train-fitting-property` but omits `train-fitting-population`). `grep train-fitting-population doc/` finds no toctree reference, so the page is unreachable from navigation.
## Stale / now-inaccurate docs to fix
### 18. Hessian freeze claims are outdated
- `doc/model/train-energy-hessian.md:94` and `doc/model/overall.md:61` claim Hessian models cannot be frozen, which `dp freeze --hessian` (JAX) now contradicts (see item 6).
## Already documented (no action needed)
DPA4/SeZM model (`doc/model/dpa4.md`), NeighborGraph graph-native inference route (`doc/model/dpa2.md:94`, `doc/model/dpa4.md:499`), multi-rank LAMMPS inference, DPA-ADAPT toolkit (`doc/dpa_adapt/overview.md`), JAX-MD interface (`doc/third-party/jaxmd.md`), atomic charge population fitting (content good — only the toctree entry is missing, see item 17), property fitting, `shared_dict` in multi-task, `dp embed` embedding sub-CLI, plugin mechanism / entry-point loading, LAMMPS dynamic backend loading, `dE/dfparam` to LAMMPS via finite differences, PyTorch custom ops enabled by default, WSD LR scheduler, charge-spin LAMMPS inference keyword.
## Notes
- **Internal/optimization-only (probably should not be user-documented):** backend-independent trainer abstraction (`55d7e7991`, `deepmd/dpmodel/`); adaptive Array API neighbor search (`2a47cdc11` — internal dense/cell-list dispatcher); `feat(core): batch prod env mat over frames` (`5e80487fe`).
- **Experimental/partial:** JAX DPA4 descriptor (`c202c7b38`) is descriptor-only (no fitting/model/trainer), so `dpa4.md`'s "PyTorch only" statement is still accurate for end-to-end training; not a doc gap yet. The dipole-charge adapter commit `31221382c` extends an already-documented modifier (`doc/model/dplr.md`) to dpmodel/JAX/TF2 backends — the modifier itself is documented.
- **Schema vs. prose caveat:** the "thinly documented" items appear in the auto-generated argument reference built by sphinx-dargs, but have no explanatory prose, examples, or guide-page coverage.
## Documentation checklist
### Fully undocumented features
- [ ] Document the TF2 eager backend and `dp --tf2` training/freeze/compress workflow (add to `doc/backend.md`, `doc/train/training.md`, `doc/freeze/freeze.md`, `doc/freeze/compress.md`) — `73de44b1f`, `26ad0fb83`
- [ ] Introduce the `pt_expt` (PyTorch-Exportable) backend in `doc/backend.md` and document its distinct capabilities (`.pt2`/AOTInductor export, LMDB datasets, multi-task, `dp finetune`/`change-bias`/`compress`) — `9bd1f16b1` et al.
- [ ] Add a JAX tab to `doc/train/training.md` and document `dp --jax train` — `20874166a`
- [ ] Document `dp test` stress output (`.s.out` file, Stress MAE/RMSE in eV/ų) in `doc/test/test.md` — `330fa7558`
- [ ] Add `serialization-tree` to the list of `dp show` choices in `doc/model/show-model-info.md` — `e19bd188b`
- [ ] Document `dp freeze --hessian` (JAX) in `doc/freeze/freeze.md` — `44131151f`
- [ ] Document the vesin neighbor list and `nlist_backend` (`auto`/`vesin`/`native`) pluggable strategy in `doc/inference/python.md` — `23622361d`
### Thinly documented features (add prose / examples)
- [ ] Extend `doc/freeze/compress.md` to cover dpmodel/JAX compression (`dp --dp compress`, `dp --jax compress`) — `f143171cf`
- [ ] Document the charge-spin training config (`add_chg_spin_ebd`, `default_chg_spin`, `dim_chg_spin`) in `doc/model/dpa3.md` and `doc/model/train-energy-spin.md` — `e3fda4537`, `f39a08128`
- [ ] Document `stat_file` / `stat_file_mode` (TF training) in `doc/train/` — `2b3bb05a7`
- [ ] Document the `so3_readout` DPA4/SeZM option in `doc/model/dpa4.md` — `4b6506db4`
- [ ] Document EMA (`enable_ema`, `ema_decay`, `ema_ckpt_keep`) in `doc/train/training-advanced.md` — `57f870ff4`
- [ ] Document `force_l2_norm` in `doc/model/train-energy.md` loss section — `3f52fa95a`
- [ ] Document `huber_delta` in the loss section — `14c349b42`
- [ ] Document `distinguish_types` (element type-wise bias) in `doc/model/train-fitting-property.md` — `7cd0d191b`
- [ ] Document LMDB dataset support in `doc/data/` and `doc/train/` — `565f4beec`, `54f42d90c`
### Orphaned / navigation-broken
- [ ] Add `train-fitting-population` to the `doc/model/index.rst` toctree
### Stale / now-inaccurate docs to fix
- [ ] Update `doc/model/train-energy-hessian.md:94` and `doc/model/overall.md:61` — Hessian models can now be frozen via `dp freeze --hessian` (JAX)
---
Coding agent: opencode
opencode version: 1.18.8
Model: ustc/glm-5.2
Reasoning effort: max
Contributor guide
Research direction
Start with one checklist item, reading its named implementation entry point alongside the target documentation file, such as deepmd/backend/tf2.py and doc/backend.md. Update the relevant guide pages, examples, and stale statements, then verify that doc/model/index.rst exposes the population-fitting page and that each selected feature is described where the checklist specifies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch, tensorflow
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100