huggingface / huggingface/torchMoji
attention weights should be reordered as well as the outputs in case inputs were sorted and packed when doing batch inference
- Dominant language
- Python
- Stars
- 920
- Forks
- 185
- PR merge metrics
- No merged PRs in 30d
Description
https://github.com/huggingface/torchMoji/blob/198f7d4e0711a7d3cd01968812af0121c54477f8/torchmoji/model_def.py#L243
during inference in batches, if `reorder_output` becomes `True` (if `input_seqs` are not of `PackedSequence`), then outputs are correctly reordered to match input order.
**however**, if `return_attention` is requested (set to `True`) then it is (and that's the bug) returned in an order that does not match the inputs.
a possible fix could be in the following section in the code:
https://github.com/huggingface/torchMoji/blob/198f7d4e0711a7d3cd01968812af0121c54477f8/torchmoji/model_def.py#L243
similarly to how outputs are reordered - we can add the attention reordering code under that same `if` statement:
```
reordered_for_weights = Variable(att_weights.data.new(att_weights.size()))
reordered_for_weights[perm_idx] = att_weights
att_weights = reordered_for_weights
```
does that make sense?
am i missing anything?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.