aims-umich / aims-umich/pyMAISE

Parallelization of `pyMAISE.PostProcessor`

Ouverte
#78 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
documentation enhancement
Langage dominant
Python
Étoiles
33
Forks
14
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

## Background
The neural networks and classical models can be trained in parallel in the `pyMAISE.Tuner`. The resulting best models are then returned for the `pyMAISE.PostProcessor` which is not parallelized yet. We need an implementation that copies or maybe even utilizes the code used in the tuner such that all models may be trained at once (if they all fit in memory). This will greatly reduce computational time for the post processor.

Files that will help with this implementation:
- `pyMAISE/utils/nn_tuner.py`
- `pyMAISE/utils/device.py`
- `pyMAISE/utils/process_pool.py`
- `pyMAISE/utils/trial.py`

Those may be used in their current implementation, extended to support the postprocessor, or examples for new code for the post processor. The former two options are preferred. One thing to note is the parallelization on GPU. If you allow tensorflow to initialize the GPU(s) on the host process then subprocesses will throw errors. You must launch subprocesses which each on their own allow tensorflow to initialize the GPU(s). Refer to the above implementations for examples on how this works.

## Tasks
- [ ] Classical and NNs trained in postprocessor are parallelized on CPU and GPU.
- [ ] We demonstrate this on MIT reactor.

## Definition of Done
The above tasks are done.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.