facebookresearch / facebookresearch/coconut
why MyCollator left pad and right pad the features?
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 189
- PR merge metrics
- No merged PRs in 30d
Description
From what I understand `__call__` function will return `---xxxxxxxx---` where `x` is word token and `-` is pad token.
The `__call__` function documentation
```
"""
Pad the batch like this to maximize the reuse of kv cache.
E.g.,
xxxxxxxxxxxxxxx--
-----xxxxxxxxxxxxx-------
---xxxxxxxxxxxxxx
("x" is word token, "-" is pad token)
"""
```
https://github.com/facebookresearch/coconut/blob/d1c10b134ad2bded92b0eca4c0a422c513a12dea/dataset.py#L79-L185
Contributor guide
Research direction
Start in dataset.py around the documented __call__ function at lines 79–185 and compare the implementation with the padding example in the issue. Determine whether the behavior is intentional and whether the documentation needs clarification; done means the discrepancy has a clear explanation or the affected documentation is corrected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100