GAP-LAB-CUHK-SZ / GAP-LAB-CUHK-SZ/Total3DUnderstanding

MGNet pretraining goes wrong

Open
#15 15 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
445
Forks
52
PR merge metrics
No merged PRs in 30d

Description

Hi Yinyu:

I tried to pretrain MGNet with `python main.py configs/mgnet.yaml --mode train` and test it with `python main.py configs/mgnet.yaml --mode test`.

However, after 50 epochs of training, the learning rate quickly reduced to a seemingly unreasonable level of 1e-08 with the best chamfer_loss stuck at 5.67 after the 6th epoch.
[log.txt](https://github.com/yinyunie/Total3DUnderstanding/files/5272828/log.txt)

Also, the test results of the best checkpoint looks like below:
[log.txt](https://github.com/yinyunie/Total3DUnderstanding/files/5272830/log.txt)

Is there anything I missed?

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the commands `python main.py configs/mgnet.yaml --mode train` and `python main.py configs/mgnet.yaml --mode test`, then inspect `configs/mgnet.yaml` and both linked `log.txt` files. Trace the learning-rate reduction, stalled chamfer loss, and best-checkpoint test output; done means the training behavior has an identified cause and the resulting checkpoint produces validated test results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.