allenai / allenai/longformer

Please describe how to use BERT or ELECTRA Models for Longformer training.

Aberta
#86 10 comentários 11 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
2.2k
Forks
286
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

Hi,
I am interested in traing a german longformer model. But there is no german RoBERTa. I have seen that it is possible to use BERT as a basis but you only have a description to use RoBERTa. Since there seems to be some demand:

1. https://github.com/allenai/longformer/issues/85
2. https://github.com/allenai/longformer/issues/21
3. https://github.com/allenai/longformer/issues/82
4. https://github.com/allenai/longformer/issues/77
5. https://github.com/allenai/longformer/issues/72
6. https://github.com/allenai/longformer/issues/65
7. https://github.com/allenai/longformer/issues/61

Could you please add a 2nd jupyter notebook on how to use BERT as basis or describe in detail what to change on the existing notebook for RoBERTa? That would realy be awsome.

Thanks
Philip

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.