huggingface / huggingface/torchMoji
loss don't go down when finetune and about requires_grad option in backward()
- Dominant language
- Python
- Stars
- 920
- Forks
- 185
- PR merge metrics
- No merged PRs in 30d
Description
I tried finetuning with SS-Youtube dataset on `examples/finetune_youtube_last`. but There are some minor errors in there :(...
First. The loss are not decrease well about each epochs when finetune using SS-Youtube. I tried training as setting original val loss 0.01, but Each epochs loss almost same
```
1 Epoch val acc 62.54 val loss 0.67718
2 Epoch val acc 62.54 val loss 0.67718
3 Epoch val acc 62.54 val loss 0.67718
...
```
Second. In finetune.py, There is error `element 0 of tensors does not require grad and does not have a grad_fn` if i not include `loss = Variable(loss, requires_grad=True)` in code.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.