zhanghang1989 / zhanghang1989/PyTorch-Encoding
问题一:print(model)与forward中定义的model不一致,与文章中的结构不一致。问题二:且训练时,无法更新fc层梯度
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 448
- PR merge metrics
- No merged PRs in 30d
Description
你好,打扰了,以下是我的问题。
训练:CUDA_VISIBLE_DEVICES=0,1,2,3 python train_dist.py --dataset minc --model deepten_resnet50_minc --batch-size 2 --lr 0.004 --epochs 80 --lr-step 60 --lr-scheduler step --weight-decay 5e-4
问题一:print(model)与forward中定义的model不一致,与文章中的结构不一致
print(model)比deepten.py中的define forward的结构多了全连接层,如下附件
print(model).txt
deepten.txt
而文章中这个结构似乎没有全连接层?
问题二:训练时,fc层梯度无法更新?
如下是我训练打印出的梯度
gra.txt
总的来说我不太清楚是否应该包含全连接层2048,1000。如果应该包含,我如何解决梯度更新的问题,如果不该包含,我该在哪里取删除它?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported command in train_dist.py and inspect the model structure and forward definition in deepten.py. Compare the printed model, deepten.py, and the attached gradient output to determine whether the 2048-to-1000 fully connected layers are expected and why their gradients do not update; the issue is done when both points have a verified explanation and resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100