allenai `LongformerEncoderDecoderForConditionalGeneration` vs huggingface `LEDForConditionalGeneration`
- Dominant language
- Python
- Stars
- 2.2k
- Forks
- 285
- PR merge metrics
- No merged PRs in 30d
Description
I would like to be able to load a LED model into huggingface via e.g.
```led = LEDForConditionalGeneration.from_pretrained('PATH/longformer-encdec-large-16384', gradient_checkpointing=True, use_cache=False)```
However, I receive an AttributeError `Can't set attention_probs_dropout_prob with value 0.0 for LEDConfig`. This occurs with both my custom LED model that I converted via `convert_bart_to_longformerencoderdecoder.py` and with the LED model `longformer-encdec-large-16384` downloaded directly from this repo. This occurs with HuggingFace transformers==4.5.1.
How can I use the LED models with the transformers library? I realize that there is a model `"allenai/led-large-16384"` in the repo already, but I would like to be able to use my custom LED model with the transformers codebase. Thanks!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.