ISISNeutronMuon / ISISNeutronMuon/MDMC

GPR minimiser has very high memory usage

Open
#1,490 1 comment 0 reactions 0 assignees Claimed by @AdamG412 View on GitHub
bug P4
Dominant language
Python
Stars
4
Forks
0
Avg merge
2d 4h
Merged PRs (30d)
5

Description

**Description of the error**
When using a GPR minimiser, memory usage is significantly higher than the other minimisers for the same number of parameters and steps. For example, with 4 parameters and 5 steps MDMC is killed with an out of memory error (on a machine with 16GB of RAM), whereas GPO and MMC can handle 100+ parameters and 100 steps.

The memory usage comes down to line 302 in `MDMC/refinement/minimizers/GPR.py`:
`point_array = list(itertools.product(*predictive_coordinates))`
Here a product of predictive coordinates is constructed and cast to a list. Each array of coordinates contains 100 elements by default, so `point_array` quickly becomes very large.

**Suggested fix**
If possible the cast to `list` should be removed and `point_array` kept as an iterator. `point_array` is later passed to the GPR's predict method, so this may not be possible.
Alternatively, the predictions could be batched.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.