ISISNeutronMuon / ISISNeutronMuon/MDMC
Performance improvements in S(Q,w) calculations
- Dominant language
- Python
- Stars
- 4
- Forks
- 0
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 5
Description
**Is your feature request related to a problem? Please describe.**
While comparing the results of MDMC and MDANSE analysis on the same trajectory, I noticed a difference in the performance of the two codes. Using a trajectory with 20450 steps and 480 atoms, the same analysis (using the same number of Q vectors) would take:
MDANSE: 2 min. coherent + 7 min. incoherent S(Q,w)
MDMC: over 20 min. for total S(Q,w)
In both cases I used parallelisation over 4 CPU cores. MDANSE parallelises using processes (since the trajectory is loaded from an HDF5 file and the data do not have to be pickled), which MDMC uses a threadpool and stores the trajectory entirely in memory.
**Describe the solution you'd like**
Profiling shows that most of the time is spent in
`MDMC/trajectory_analysis/observables/fqt.py - calculate_rho`
which at the moment performs each calculation for a single q vector. The `for` loop could be removed from that function since the np.dot function is able to perform the multiplication for an entire array of q vectors at the same time.
**Describe alternatives you've considered**
Once the H5MD writeout has been added to the main branch, it will become possible to read the data from file in separate processes similar to the way it is done in MDANSE. However, it would require more programming effort and the performance benefit could only be estimated afterwards.
**Additional context**
N/A
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with MDMC/trajectory_analysis/observables/fqt.py, focusing on calculate_rho, and profile the S(Q,w) calculation using the trajectory and four-core setup described in the issue. Done means the calculation produces equivalent results while reducing runtime for multiple q vectors; compare both outputs and performance against the current behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100