codertimo / codertimo/BERT-pytorch
Why Segment Embedding number only 3?
Open
- Dominant language
- Python
- Stars
- 6.5k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
```
import torch.nn as nn
class SegmentEmbedding(nn.Embedding):
def __init__(self, embed_size=512):
super().__init__(3, embed_size, padding_idx=0)
```
This is the source code. First idx is padding, thus only 2 segment is supported. Why does Bert support 2 segments only?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.