Train: Some weights of the model checkpoint at openai/clip-vit-large-patch14 were not used when initializing CLIPTextModel

When running training with the exact settings as the tutorial fill50k dataset, I got this:
got this:

Some weights of the model checkpoint at openai/clip-vit-large-patch14 were not used when initializing CLIPTextModel: ['vision_model.encoder.layers.9.self_attn.q_proj.weight', 'vision_model.encoder.layers.16.layer_norm2.bias', ''vision_model.encoder.layers.0.layer_norm2.weight', 

- This IS expected if you are initializing CLIPTextModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing CLIPTextModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

Is that expected?

thanks!

Reproduce the warning using the tutorial's fill50k dataset settings and the openai/clip-vit-large-patch14 checkpoint. Start by tracing where CLIPTextModel is initialized and compare the reported unused vision_model weights with the model architecture; done means establishing whether the warning is expected or identifies a training configuration problem.