allenai / allenai/sequential_sentence_classification

Implementation of segmentation embeddings

Open
#11 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
77
Forks
28
PR merge metrics
No merged PRs in 30d

Description

Hi,
Thanks for releasing this awesome repo.
In your implementation, I found each input sentence has a separate id in the "bert-type-ids". I wonder if you use these sentences ids to generate the segmentation embeddings?

BERT uses the sum of the token embeddings, the segmentation embeddings, and the position embeddings as input embeddings. They define there are no more than two input segments and only use 0 and 1 as the segment ids, which means the segment ids greater than 1 have not appeared in the pre-training of BERT.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.