huggingface / huggingface/transformers
Truncating the prefix of a sequence rather than the suffix
- Dominant language
- Python
- Stars
- 166k
- Forks
- 34.6k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 281
Description
# 🚀 Feature request
Hi, tokenizers get `truncation` as an argument. When set to `True` the tokenizer will truncate the suffix of a sequence so it does not surpass the specified `max_length`. I'd like to have a functionality that truncates the prefix of the sequence, so the model will see the suffix of the sequence.
## Motivation
In many applications (e.g. Dialog, and QA) the most important part of the sequence is the suffix (e.g. the question after the context, or the last response of the dialog).
## Your contribution
Perhaps I'll submit a PR, but it might take me some time as I'm close to some deadlines of mine :(
Contributor guide
Research direction
Start by tracing the tokenizer truncation argument through the Transformers tokenizer entry points and existing truncation tests. Determine how prefix truncation should interact with max_length and the existing suffix-truncation behavior. Done means a supported option preserves the sequence suffix and tests cover the requested dialog and question-after-context use cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100