huggingface / huggingface/transformers
Dependency parsing head for pretrained models
- Dominant language
- Python
- Stars
- 166k
- Forks
- 34.6k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 281
Description
# 🚀 Feature request
Add a new classification head for pretrained models, for dependency parsing.
## Motivation
Current heads, such as `AutoModelForTokenClassification`, does not work well for finetuning a pretrained model for dependency parsing. [There are now such heads available](https://openreview.net/pdf?id=Hk95PK9le), so adding such would make it a lot easier to finetune models for such tasks. A `PyTorch` implementation of that paper (and others) can be found [in this repo](https://github.com/yzhangcs/parser).
## Your contribution
I could assist with a PR, but for now I'd like to start the discussion to see if this is something that the HF team and others would be interested in.
Contributor guide
Research direction
Start by reviewing AutoModelForTokenClassification and the linked paper and PyTorch parser repository to understand the proposed dependency-parsing head. Done means adding a pretrained-model classification head that supports fine-tuning for dependency parsing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100