deepinsight / deepinsight/insightface

[arcface_torch] train.py in arcface_torch DistributedDataParallel should wrap the module_partial_fc ?

Open
#1,430 6 comments 0 reactions 0 assignees View on GitHub
arcface_torch
Dominant language
Python
Stars
29.7k
Forks
6.1k
PR merge metrics
No merged PRs in 30d

Description

I found in the code , that the backbone in wrapped by DistributedDataParallel , so the model will sync different gradient in different gpus , but conversely the module_partial_fc isn't wrapped by DistributedDataParallel , will the center's weight in different gpu be different ? and should the center be the same in different gpus ?

@anxiangsir

```

backbone = torch.nn.parallel.DistributedDataParallel(
module=backbone, broadcast_buffers=False, device_ids=[local_rank])
backbone.train()

margin_softmax = eval("losses.{}".format(args.loss))()
module_partial_fc = PartialFC(
rank=rank, local_rank=local_rank, world_size=world_size, resume=args.resume,
batch_size=cfg.batch_size, margin_softmax=margin_softmax, num_classes=cfg.num_classes,
sample_rate=cfg.sample_rate, embedding_size=cfg.embedding_size, prefix=cfg.output)

```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.