huggingface / huggingface/torchMoji

attention weights should be reordered as well as the outputs in case inputs were sorted and packed when doing batch inference

Open
#25 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
920
Forks
185
PR merge metrics
No merged PRs in 30d

Description

https://github.com/huggingface/torchMoji/blob/198f7d4e0711a7d3cd01968812af0121c54477f8/torchmoji/model_def.py#L243

during inference in batches, if `reorder_output` becomes `True` (if `input_seqs` are not of `PackedSequence`), then outputs are correctly reordered to match input order.

**however**, if `return_attention` is requested (set to `True`) then it is (and that's the bug) returned in an order that does not match the inputs.

a possible fix could be in the following section in the code:
https://github.com/huggingface/torchMoji/blob/198f7d4e0711a7d3cd01968812af0121c54477f8/torchmoji/model_def.py#L243
similarly to how outputs are reordered - we can add the attention reordering code under that same `if` statement:
```
reordered_for_weights = Variable(att_weights.data.new(att_weights.size()))
reordered_for_weights[perm_idx] = att_weights
att_weights = reordered_for_weights
```

does that make sense?
am i missing anything?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.