DDMAL / DDMAL/Calvo_classifier
Wishlist for settings and ports of Fast Calvo Trainer (all versions of the job)
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Wish list for Patchwise Trainer (for all versions of the job—HPC and non-HPC one):
- [ ] log file as an output port
This is present in the non-hpc version, but still missing in the HPC one. The logs are useful for evaluating the performance of the job: how far it went in terms of epochs over the maximum number of epochs given to the job and how many samples it ended up taking over the maximum number of samples given. This helps to modify the job settings to get better results and to realize whether to invest more time in preparing more training data or not.
- [ ] an early stop setting
By default, the early stop parameter is set to 3. This is encoded by `patience = 3` in the training_engine_sae.py ([here](https://github.com/DDMAL/Calvo_classifier/blob/develop/training_engine_sae.py#L216)). This means that the job will stop training a model either when it reaches the maximum number of epochs for that model (default = 15), or when it hasn't improved the performance (accuracy) after 3 consecutive epochs. This has the effect that we usually never got to the maximum number of epochs. Sometimes the model is saved as soon as 4 epochs have passed. Therefore, it will be useful to edit this parameter to guarantee that we get further in the number of epochs for training the model for a longer time and improve its performance. Important restriction `early stop < maximum number of epochs`
- [ ] Change the default of the memory setting (in HPC) to `150,000` (rather than `257,000 Megas` which is the maximum memory allowed in Cedar).
I will run some experiments to find out a minimum amount of memory that still does the job, but for now `150GB` does work and it is still better than giving it the maximum value allowed.
- [ ] Remove the batch size from the settings. It was supposed to be a constant (not a variable that the user can change) set to the value of `16` (see [here](https://github.com/DDMAL/Calvo_classifier/blob/develop/training_engine_sae.py#L29))
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.