huggingface / huggingface/transformers

Adding RelationExtraction head to layoutLMv2 and layoutXLM models

Open
#15,451 31 comments 9 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

New model
Dominant language
Python
Stars
166k
Forks
34.6k
Avg merge
3d 9h
Merged PRs (30d)
281

Description

# 🌟 New model head addition
Relation Extraction Head for LayoutLMv2/XLM
## Addition description
Hey all,

I've see a bunch of different requests across huggingface issues [[0]](https://github.com/huggingface/transformers/issues/14330), unilm issues [[0]](https://github.com/microsoft/unilm/issues/286)[[1]](https://github.com/microsoft/unilm/issues/465) and on @NielsRogge Transformer Tutorials issues [[0]](https://github.com/NielsRogge/Transformers-Tutorials/issues/6)[[1]](https://github.com/NielsRogge/Transformers-Tutorials/issues/39) about adding the relation extraction head from layoutlmv2 to the huggingface library. As the model is quite difficult to use in it's current state I was going to write my own layer ontop but I saw in this [issue](https://github.com/NielsRogge/Transformers-Tutorials/issues/39) that it may be a good idea to add it to transformers as a separate layoutlmv2/xlm head and thought it would be a good way to contribute back to a library I use so much.

I've gone ahead and added it under my own [branch](https://github.com/R0bk/transformers/tree/layoutlm-relation-extraction) and got it successfully working with the library. [Here](https://colab.research.google.com/drive/16wqA3oTUf7yzUKsSSZxiMf1443_ZO3wC?usp=sharing) is a colab using my branch of transformers if you want to test it yourself.

Before I add tests/ write more docs I just wanted to post here first to see if there's interest in potentially merging this in. If there is interest I have a few questions that it would be helpful to get some info on to ensure that I've correctly done the integration.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the layoutlm-relation-extraction branch and its linked Colab to understand the proposed LayoutLMv2/XLM relation-extraction head. The issue explicitly leaves tests and documentation to be added; completion would require maintainer agreement on integration, followed by those tests and docs.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning, python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.