codertimo/BERT-pytorch

View on GitHub

Google AI 2018 BERT pytorch implementation

Stars: 6.5k
Forks: 1.3k
Open beginner issues: 0
Indexed issues: 57

Dominant language: Python
License: Apache-2.0
Last GitHub push: Oct 30, 2018
Latest indexed: Sep 15, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted good first issue
PR merge metrics: No merged PRs in 30d

Open All

57 open issues indexed

what’s your data set? Open

question

codertimo/BERT-pytorch#1 · 7 comments · 0 reactions · 2 assignees · Indexed Sep 15, 2026
Is it possible to train BERT? Open

help wanted question

codertimo/BERT-pytorch#3 · 7 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
when training the masked LM, the unmasked words (have label 0) were trained together with masked words? Open

enhancement question

codertimo/BERT-pytorch#29 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
pred_loss decrease fast while avg_acc stay at 50% Open

help wanted invalid question

codertimo/BERT-pytorch#32 · 53 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Vocab Replace \t to blank issue Open

bug

codertimo/BERT-pytorch#33 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Pretrained model transfer to pytorch Open

enhancement help wanted

codertimo/BERT-pytorch#34 · 8 comments · 11 reactions · 1 assignee · Indexed Sep 15, 2026
how to test the model? Open

enhancement question

codertimo/BERT-pytorch#38 · 2 comments · 3 reactions · 0 assignees · Indexed Sep 15, 2026
Tie the input and output embedding? Open

enhancement

codertimo/BERT-pytorch#40 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
the format of input Open

question

codertimo/BERT-pytorch#41 · 4 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Making Wikipedia Corpus Open

help wanted

codertimo/BERT-pytorch#42 · 1 comment · 2 reactions · 0 assignees · Indexed Sep 15, 2026
Making Book Corpus Open

help wanted

codertimo/BERT-pytorch#43 · 5 comments · 2 reactions · 0 assignees · Indexed Sep 15, 2026
What should be the shapes and example of values of x and segment_info in BERT.forward? Open

codertimo/BERT-pytorch#48 · 0 comments · 6 reactions · 0 assignees · Indexed Sep 15, 2026
Question about the loss of Masked LM Open

good first issue

codertimo/BERT-pytorch#49 · 5 comments · 13 reactions · 0 assignees · Indexed Sep 15, 2026
made a script to generate bert pre-train data Open

codertimo/BERT-pytorch#50 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
IndexError: list index out of range Open

codertimo/BERT-pytorch#52 · 13 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
PositionalEmbedding Open

good first issue

codertimo/BERT-pytorch#53 · 3 comments · 4 reactions · 0 assignees · Indexed Sep 15, 2026
will "random.randint(self.corpus_lines if self.corpus_lines < 1000 else 1000)" report error? Open

codertimo/BERT-pytorch#55 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
chooses 15% of token Open

good first issue

codertimo/BERT-pytorch#56 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 15, 2026
How to test your model after training on my own dataset? Open

codertimo/BERT-pytorch#57 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
self.d_k = d_model // h gives 64 dimension ? Open

codertimo/BERT-pytorch#60 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
How can I finetune with this project Open

codertimo/BERT-pytorch#61 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
OOM error in cuda while passing large corpus of wikipedia text files ? how to manage big files to train Open

codertimo/BERT-pytorch#64 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Erroneous Code Open

codertimo/BERT-pytorch#65 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
ONNX conversion: TransformerBlock problem Open

codertimo/BERT-pytorch#67 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Vocab's load_vectors seems to be an old method? Open

codertimo/BERT-pytorch#69 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
# of parameter is larger than bert base instead im using specs less than bert base Open

codertimo/BERT-pytorch#70 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Did you have tried to use ipdb to set breakpoint in __get__item__, which is a funtion from BERTDataset Open

codertimo/BERT-pytorch#71 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
predict next sentence ony? it also can do SQA assignments? Open

codertimo/BERT-pytorch#72 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Can you share me a trained data? Open

codertimo/BERT-pytorch#73 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
How to use BERT model to fine-tune a cloze-style task? Open

codertimo/BERT-pytorch#74 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
If I want to use /u as a placeholder instead of /t, what do I need to do Open

codertimo/BERT-pytorch#75 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
How to Output Embedded Sentence Vector? Open

codertimo/BERT-pytorch#76 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
embedding/position.py with RuntimeError if d_model is odd Open

codertimo/BERT-pytorch#78 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
How to use this bert model to load and finetune ? Open

codertimo/BERT-pytorch#79 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
How to implement model once pretrained for masked input sentences? Open

codertimo/BERT-pytorch#80 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Default model sizes are much smaller than BERT base Open

codertimo/BERT-pytorch#81 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
how to do Ner Open

codertimo/BERT-pytorch#82 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
transformer.py 中的forword方法调用的SublayerConnection类。实现残差链接和标准化的实现 Open

codertimo/BERT-pytorch#83 · 2 comments · 3 reactions · 0 assignees · Indexed Sep 15, 2026
In Next Sentence Prediction task,the original code may choose the same line when you try to use the negative sample Open

codertimo/BERT-pytorch#86 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
An error occurred【AttributeError: type object 'BERT' has no attribute 'hidden'】 Open

codertimo/BERT-pytorch#87 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
IndexError Open

codertimo/BERT-pytorch#88 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
The exact English pretraining data and Chinese pretraining data that are exact same to the BERT paper's pretraining data. Open

codertimo/BERT-pytorch#89 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
why language_model.py has different vectors Open

codertimo/BERT-pytorch#91 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Why not use torch.no_grad when evaluating test data? Open

codertimo/BERT-pytorch#92 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
dataset / dataset.py have one erro? Open

codertimo/BERT-pytorch#93 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
It keeps trying to use CUDA despite --with_cuda False option Open

codertimo/BERT-pytorch#94 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Pooler layer? Open

codertimo/BERT-pytorch#95 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
bert-vocab? Open

codertimo/BERT-pytorch#96 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
why specify `ignore_index=0` in the NLLLoss function in BERTTrainer? Open

codertimo/BERT-pytorch#98 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 15, 2026
What dataset did you use to train model? Open

codertimo/BERT-pytorch#99 · 2 comments · 3 reactions · 0 assignees · Indexed Sep 15, 2026

codertimo/BERT-pytorch

Get new issues in your inbox