zhanghang1989 / zhanghang1989/PyTorch-Encoding

How can I use TripletMarginLoss with DataParallelCriterion?

Open
#184 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2k
Forks
448
PR merge metrics
No merged PRs in 30d

Description

I'd like to use triplet loss with DataParallelCriterion as follows, but some errors came out.

model = Model()
model = DataParallelModel(model).cuda()

criterion = nn.TripletMarginLoss()
criterion = DataParallelCriterion(criterion).cuda()

a_embs = model(a_node)
p_embs = model(p_node)
n_embs = model(n_node)

loss = criterion(a_embs, p_embs, n_embs)
Traceback (most recent call last):
  File "sync_train.py", line 424, in <module>
    main()
  File "sync_train.py", line 108, in main
    train(sync_train_data, model, criterion, optimizer, epoch, config)
  File "sync_train.py", line 199, in train
    loss = criterion(a_embs, p_embs, n_embs)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/myProject/parallel.py", line 140, in forward
    outputs = _criterion_parallel_apply(replicas, inputs, targets, kwargs)
  File "/myProject/parallel.py", line 194, in _criterion_parallel_apply
    raise output
  File "/myProject/parallel.py", line 168, in _worker
    output = module(*(input + target), **kwargs)
TypeError: add() received an invalid combination of arguments - got (tuple), but expected one of:
 * (Tensor other, Number alpha)
 * (Number other, Number alpha)

I already tried this, but it doesn't work for me.

Could you help me?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with /myProject/parallel.py, especially DataParallelCriterion.forward and _criterion_parallel_apply, then reproduce the shown TripletMarginLoss call using the traceback's setup. Check how the three embedding arguments are combined for each replica; done means the criterion call runs without the tuple-related TypeError and preserves the intended triplet-loss inputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.