Longformer model with weight(model.encoder.embed_positions.weight) error
- Vorherrschende Sprache
- Python
- Sterne
- 2.2k
- Forks
- 286
- PR-Merge-Kennzahlen
- Keine gemergten PRs in 30 T.
Beschreibung
```bash
RuntimeError: Error(s) in loading state_dict for BartModel:
size mismatch for model.encoder.embed_positions.weight: copying a param with shape torch.Size([16386, 768]) from checkpoint, the shape in current model is torch.Size([1026, 768]).
```
I use longformer model called longformer-encdec-base-16384 which is downloaded in https://github.com/allenai/longformer,and use huggingface to load the model,when transformers’ version is 3.1.0, the code can run, but when it is 4.4.2,the error happened.
MeanWhile,when I use the model to proposal pairs of sentences,I found it that the returned token_type_ids values are just zero
without one. how ever,in the model's special_tokens_map.json, it has defined cls_token and sep_token.
Finally, I sincerely hope you would reply me soon. Thanks!
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.