autogluon / autogluon/autogluon

[AutoMM] Configurable ray chekpoints in HPO

Open
#4,100 1 comment 0 reactions 1 assignee Claimed by @FANGAreNotGnu View on GitHub
enhancement feature: hpo module: multimodal
Dominant language
Python
Stars
10.7k
Forks
1.2k
Avg merge
21h 29m
Merged PRs (30d)
57

Description

## Description

Right now, it is really hard to perform a HPO with transformer models given that a great amount of checkpoints is created, resulting is TBs of storage needed only to make a single HPO.

Specifically I've had troubles with MultiModalPredictor with a NER task.

I think this would be simpler if the parameters below could be configured:

https://github.com/autogluon/autogluon/blob/949a7815717e48f4675fcf079db0455fd509e444/multimodal/src/autogluon/multimodal/utils/hpo.py#L174

However, ideally there whould be some kind of checkpoint cleaner that deletes the pkl of models that did not achieve a good score and will not be used in inference. Simply keeping a config file for the statistical data will suffice

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.