huggingface / huggingface/transformers

Dependency parsing head for pretrained models

Open
#13,355 8 comments 1 reaction 0 assignees View on GitHub
Feature request
Dominant language
Python
Stars
166k
Forks
34.6k
Avg merge
3d 9h
Merged PRs (30d)
281

Description

# 🚀 Feature request

Add a new classification head for pretrained models, for dependency parsing.

## Motivation

Current heads, such as `AutoModelForTokenClassification`, does not work well for finetuning a pretrained model for dependency parsing. [There are now such heads available](https://openreview.net/pdf?id=Hk95PK9le), so adding such would make it a lot easier to finetune models for such tasks. A `PyTorch` implementation of that paper (and others) can be found [in this repo](https://github.com/yzhangcs/parser).

## Your contribution

I could assist with a PR, but for now I'd like to start the discussion to see if this is something that the HF team and others would be interested in.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing AutoModelForTokenClassification and the linked paper and PyTorch parser repository to understand the proposed dependency-parsing head. Done means adding a pretrained-model classification head that supports fine-tuning for dependency parsing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.