Webdatasets for Linear Probe evaluation

Hi, I noticed only the following web datasets work for the linear probe evaluation. It would be nice to have the following things fixed for the same:

1. Support for the rest of the datasets to do linear probing evaluations. For example, the SUN397, the train shard file is missing from the HuggingFace datasets.

2. Similar to zero-shot evaluation, an automatic way to run the linear probe evaluations would be nice to benchmark the models quickly.

Web datasets that worked for linear probe evaluation:

wds/imagenet1k
wds/fer2013
wds/voc2007
wds/cars
wds/fgvc_aircraft
wds/mnist
wds/stl10
wds/gtsrb
wds/renderedsst2
wds/vtab/caltech101
wds/vtab/cifar10
wds/vtab/cifar100
wds/vtab/clevr_count_all
wds/vtab/clevr_closest_object_distance
wds/vtab/diabetic_retinopathy
wds/vtab/dmlab
wds/vtab/dsprites_label_orientation
wds/vtab/dsprites_label_x_position
wds/vtab/dsprites_label_y_position
wds/vtab/dtd
wds/vtab/eurosat
wds/vtab/kitti_closest_vehicle_distance
wds/vtab/flowers
wds/vtab/pets
wds/vtab/pcam
wds/vtab/resisc45
wds/vtab/smallnorb_label_azimuth
wds/vtab/smallnorb_label_elevation
wds/vtab/svhn

Thanks! :)

Start by locating the existing linear-probe evaluation path and the automatic zero-shot evaluation flow, then inspect how WebDataset sources such as the HuggingFace SUN397 dataset are discovered. Done means the remaining datasets can run linear probing and an automatic benchmark command is available for these evaluations.