allenai / allenai/longformer

allenai `LongformerEncoderDecoderForConditionalGeneration` vs huggingface `LEDForConditionalGeneration`

Open
#192 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.2k
Forks
285
PR merge metrics
No merged PRs in 30d

Description

I would like to be able to load a LED model into huggingface via e.g.

```led = LEDForConditionalGeneration.from_pretrained('PATH/longformer-encdec-large-16384', gradient_checkpointing=True, use_cache=False)```

However, I receive an AttributeError `Can't set attention_probs_dropout_prob with value 0.0 for LEDConfig`. This occurs with both my custom LED model that I converted via `convert_bart_to_longformerencoderdecoder.py` and with the LED model `longformer-encdec-large-16384` downloaded directly from this repo. This occurs with HuggingFace transformers==4.5.1.

How can I use the LED models with the transformers library? I realize that there is a model `"allenai/led-large-16384"` in the repo already, but I would like to be able to use my custom LED model with the transformers codebase. Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.