huggingface / huggingface/transformers
Siamese Multi-depth Transformer-based Hierarchical Encoder
- Dominant language
- Python
- Stars
- 166k
- Forks
- 34.6k
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 276
Description
# 🌟 New model addition
## Model description
Recently Google is published paper titled ["Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching"](https://arxiv.org/abs/2004.12297). And according to paper for long-form document matching SMITH model outperforms the previous state-of-the-art models including hierarchical attention, multi-depth attention-based hierarchical recurrent neural network, and BERT.
I feel it is will add value to already awesome transformers models collection :slightly_smiling_face:
## Open source status
* [X] the model implementation is available: https://github.com/google-research/google-research/tree/master/smith
* [X] the model weights are available: [SMITH-WP+SP model checkpoint](http://storage.googleapis.com/gresearch/smith_gwikimatch/smith_wsp_pretrain_ckpt_opensource.zip) and [GWikiMatch data](http://storage.googleapis.com/gresearch/smith_gwikimatch/gwikimatch_open_source.zip)
* [X] who are the authors: https://github.com/yangliuy, https://github.com/eladeban
Contributor guide
Assessment
This issue has not been assessed yet.