deepmodeling / deepmodeling/DeePTB
Stabilizing Training with Small Batch Sizes using Exponential Moving Average (EMA)
- Dominant language
- Python
- Stars
- 122
- Forks
- 36
- PR merge metrics
- No merged PRs in 30d
Description
### Background
Small batch sizes are commonly used in Hamiltonian AI training, as seen in [the universal DeepH](https://doi.org/10.1016/j.scib.2024.06.011). However, this approach can lead to violent updates of model parameters.
Specific Observation (DeepTB on QH9 Dataset with Batch Size=1):

### Describe the solution you'd like
Address the existing [TODO](https://github.com/deepmodeling/DeePTB/blob/57261bb10e5df0b26b9e0170c1becf31e954cc9f/dptb/nnops/trainer.py#L137) regarding EMA implementation to ensure consistent training behavior across related projects.
### Additional Context
This approach is already integrated into [QHNet](https://github.com/divelab/AIRS/blob/e41a755aba8b74f9f6d4372e0e9d654da38b1343/OpenDFT/QHBench/QH9/main.py#L11C23-L11C47) as a default setting.
Contributor guide
Research direction
Start at the EMA TODO in dptb/nnops/trainer.py around line 137, then compare the linked QHNet implementation in OpenDFT/QHBench/QH9/main.py. Determine the expected EMA behavior for small-batch training and how it should fit the existing trainer flow. Done means the TODO is implemented and training behavior is consistent with the stated EMA approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100