distilbert tensor size issue .
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python, pytorch
- Domain
- machine-learning
Research direction
Start at the training loop's model(input_ids, attention_mask=..., labels=...) call and inspect the shapes of input_ids, attention_mask, and labels before the forward pass. Compare those dimensions with the reported 512 limit and confirm the training loop completes without the tensor-size error.
Written by the indexing model from the issue text.
Description
for epoch in range(NUM_EPOCHS):
model.train()
for batch_idx, batch in enumerate(train_loader):
### Prepare data
input_ids = batch['input_ids'].to(DEVICE)
attention_mask = batch['attention_mask'].to(DEVICE)
labels = batch['labels'].to(DEVICE)
### Forward
outputs = model(input_ids, attention_mask=attention_mask, labels=labels) **
loss, logits = outputs['loss'], outputs['logits']
this is returning error :
RuntimeError: The size of tensor a (3157) must match the size of tensor b (512) at non-singleton dimension 1
i did everything just like the book says
- Dominant language
- Jupyter Notebook
- Stars
- 5.4k
- Forks
- 1.8k
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from rasbt/machine-learning-book
-
Difficulty 1/5 Under an hour Newbie friendliness 65/100
rasbt/machine-learning-book#238 ·
-
Chapter 19, page 681 Open
Difficulty 2/5 1-3 hours Newbie friendliness 52/100
rasbt/machine-learning-book#240 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 55/100
rasbt/machine-learning-book#228 ·
-
Chapter 19, Page 681 Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
rasbt/machine-learning-book#213 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
rasbt/machine-learning-book#199 · 14 comments ·
All issues in rasbt/machine-learning-book
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
bancolombia/sentinel#21 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
infer-actively/pymdp#454 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
UKGovernmentBEIS/inspect_evals#2486 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100