Smaller scale than "small"?

Is it possible to run experiments with a smaller scale than the "small" scale? It still has 12.8M samples in the candidate pool and needs 4xA100 GPU hours to train the model on a candidate subset.

I believe a smaller scale provides value for several reasons:

• It allows researchers with low compute resources to run experiments.
• It allows to try out much more subset selection strategies and/or hyper parameter search across parameters of subset selection strategies.

To address the risk of smaller scales not providing enough signal: An "xsmall" scale will probably still provide a good indication of performance on larger scales. Your paper has shown, citing from it:

Improving downstream performance at smaller scales correlates positively with
performance gains at larger scales. These trends suggests that dataset filtering can be studied
effectively at smaller scales, even with less computational resources.

Happy to also create a PR to add the additional scale to this repo. E.g. as an "xsmall-experimental" scale which is 10x smaller than "small".

No files, tests, or entry points are identified. Start by locating how the existing "small" scale is defined and determine what an "xsmall-experimental" scale should include; done means the additional scale is available for running experiments at substantially lower resource requirements.