huggingface / huggingface/setfit

Does num_iterations create duplicate data?

Open
#126 0 comments 4 reactions 0 assignees View on GitHub
question
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
267
Avg merge
36m
Merged PRs (30d)
5

Description

I am trying to get a better understanding behind this hyperparam. As far as I understand, you are iterating over the data `num_iterations` times and create a positive and negative pair by sampling. Could this result in duplicate data?

Also sometimes it tends to result in more examples than potential pairs for example in `imdb` for 3 shot there are 6 examples, 2 per class. Setting `num_iterations` to 5 creates 6 (examples) * 2 (1 positive + 1 negative) * 5 (num_iterations) = 60 examples. The possible combinations though are 6*6/2-6 = 12, essentially half of the matrix of all pairs without the diagonal.

If the above is correct it seems that its like running training for multiple epochs. Is that right? If so, why are you not creating all pairs instead and keep the `epochs` hyperparam as is which might be more intuitive. If you want a way to sample less data, why not introduce a `sample_size` to cap those combinations to a lesser number for experimentation?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.