pytorch / pytorch/vision

Segmentation reference train script

Open
#1,148 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

awaiting response module: models module: reference scripts needs reproduction topic: semantic segmentation
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

I am training deeplabv3 model from torchvision for 91 classes of coco. I am using training script from references. I use following command to train the model.
python -m torch.distributed.launch --nproc_per_node=4 --use_env train.py --dataset coco --model deeplabv3_resnet101 --output-dir output/ -b 2 -j 4

However the training gets stuck at with no error or anything
Epoch: [6] [ 8160/14525] eta: 2:14:49 lr: 0.008007956757936691 loss: 0.3471 (0.5539) time: 1.2725 data: 0.0021 max mem: 4783

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the supplied command with the referenced train.py and record the environment, worker behavior, and whether all processes stop at the reported iteration. Inspect the distributed training and data-loading paths around epoch 6; done means identifying a reproducible cause or showing that training progresses without hanging.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.