autogluon / autogluon/autogluon
Add more custom eval metric options to the MM module.
- Dominant language
- Python
- Stars
- 10.7k
- Forks
- 1.2k
- Avg merge
- 21h 29m
- Merged PRs (30d)
- 57
Description
Adding a custom eval metric to the MM predictor is almost impossible.
It's a link to the torchmetrics and there it's a way doing a custom torchmetric funtions, which then should be added to the predictor. I don't think many will succeed doing this without use many hours of work. The torchmetric has also limited default metrics to use.
How about adding additional other possible metrics, such as the one from the tabular predictor function which is compatible with common codes, even if it's not equal as fast and optimized as the torchmetric.
Contributor guide
Research direction
Start with the MM predictor's custom evaluation metric handling and its torchmetrics integration. Compare the available tabular predictor metrics with the MM module's current options, then define which additional metrics and custom-function path are in scope. Done means the selected metrics can be supplied to and used by the MM predictor.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100