model_head.pkl not loaded when HF repo is made private

Hello.

Whenever I use `SetFitModel.from_pretrained(model_id, use_auth_token=...)` to load a model from a private repo, the model_head.pkl is not found. When the repo is made public, the head is found and loaded as normal. My repo contains both `model_head.pkl` and `pytorch_model.bin` files.

Please check this please.