huggingface / huggingface/transformers

CharacterBERT

Open
#9,061 15 comments 5 reactions 0 assignees View on GitHub
New model
Dominant language
Python
Stars
166k
Forks
34.6k
Avg merge
3d 8h
Merged PRs (30d)
276

Description

# 🌟 New model addition

## Model description
**CharacterBERT** is a **variant of BERT** that uses a CharacterCNN module **instead of** WordPieces. As a result, the model:
1. Does not require/rely on a WordPiece vocabulary
2. Produces a single embedding for any (reasonable) input token
3. Is more robust to misspellings

Paper: https://www.aclweb.org/anthology/2020.coling-main.609/

## Open source status

* [x] the model implementation is available: https://github.com/helboukkouri/character-bert
* [x] the model weights are available: https://github.com/helboukkouri/character-bert/blob/main/download.py#L16
* [x] who are the authors: @helboukkouri @osf9018 @Jekub @hiroshinoji @PierreZweigenbaum and Junichi Tsujii

I am willing to work on a PR but I will probably need some guidance 😊

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.