huggingface / huggingface/setfit

Data validation when using differentiable_head

Open
#513 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
267
Avg merge
36m
Merged PRs (30d)
5

Description

Hi,

Thanks for the great package. I was training a model with the default settings, which went _great_. I then switched to a differentiable head, which crashed because I used string labels and didn't specify the number of classes. My bad! I should've read the docs.

However, I think it would be nice if it crashed immediately, and not after the fine-tuning stage. I think the dataset parameters could be checked, or even converted automatically:

* We could check the number of classes, and hence set the output layer dimensionality automatically. (unless the differentiable head is created before the call to `.train()`)
* We could definitely convert the labels to integers automatically, which is exactly what a sklearn LogisticRegression ends up doing under the hood.
* In the worst case, it would be nice to just throw a `ValueError` before starting training if the labels/classes aren't correct (i.e., the number of classes doesn't match or the labels are strings.)

I'd definitely be willing to do a PR, but only if you think it makes sense.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.