Compatibility with latest Huggingface implementation of Longformers?
- Ngôn ngữ chính
- Python
- Star
- 2.2k
- Fork
- 285
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
Hi! I've been running the `convert_model_to_long.ipynb` and works great with 🤗 transformers 3.0.2.
When upgrading to the latest version of Huggingface (3.5.1), the notebook doesn’t t work and they’ve seem to changed their [implementation](https://github.com/huggingface/transformers/issues/6465#issuecomment-673948643) of the Longformer and LongformerSelfAttention
Is there an easy way change the `RobertaLongSelfAttention` to the current Huggingface implementation or plans for updataing the notebook?
Following [hitskyer](https://github.com/allenai/longformer/issues/129), the dimensions becomes correct, but I don't know how to proceed and change the old forward pass into [This](https://github.com/huggingface/transformers/blob/c89bdfbe720bc8f41c7dc6db5473a2cb0955f224/src/transformers/models/longformer/modeling_longformer.py#L415).
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.