facebookresearch / facebookresearch/coconut

why MyCollator left pad and right pad the features?

Open
#33 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
189
PR merge metrics
No merged PRs in 30d

Description

From what I understand `__call__` function will return `---xxxxxxxx---` where `x` is word token and `-` is pad token.

The `__call__` function documentation

```
"""
Pad the batch like this to maximize the reuse of kv cache.
E.g.,

xxxxxxxxxxxxxxx--
-----xxxxxxxxxxxxx-------
---xxxxxxxxxxxxxx

("x" is word token, "-" is pad token)
"""
```

https://github.com/facebookresearch/coconut/blob/d1c10b134ad2bded92b0eca4c0a422c513a12dea/dataset.py#L79-L185

Contributor guide

Open the contributing guide

Research direction

Start in dataset.py around the documented __call__ function at lines 79–185 and compare the implementation with the padding example in the issue. Determine whether the behavior is intentional and whether the documentation needs clarification; done means the discrepancy has a clear explanation or the affected documentation is corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.