huggingface / huggingface/torchMoji

loss don't go down when finetune and about requires_grad option in backward()

Open
#21 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
920
Forks
185
PR merge metrics
No merged PRs in 30d

Description

I tried finetuning with SS-Youtube dataset on `examples/finetune_youtube_last`. but There are some minor errors in there :(...
First. The loss are not decrease well about each epochs when finetune using SS-Youtube. I tried training as setting original val loss 0.01, but Each epochs loss almost same
```
1 Epoch val acc 62.54 val loss 0.67718
2 Epoch val acc 62.54 val loss 0.67718
3 Epoch val acc 62.54 val loss 0.67718
...
```
Second. In finetune.py, There is error `element 0 of tensors does not require grad and does not have a grad_fn` if i not include `loss = Variable(loss, requires_grad=True)` in code.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.