huggingface / huggingface/transformers

Truncating the prefix of a sequence rather than the suffix

Open
#12,909 11 comments 0 reactions 0 assignees View on GitHub
WIP
Dominant language
Python
Stars
166k
Forks
34.6k
Avg merge
3d 9h
Merged PRs (30d)
281

Description

# 🚀 Feature request

Hi, tokenizers get `truncation` as an argument. When set to `True` the tokenizer will truncate the suffix of a sequence so it does not surpass the specified `max_length`. I'd like to have a functionality that truncates the prefix of the sequence, so the model will see the suffix of the sequence.

## Motivation

In many applications (e.g. Dialog, and QA) the most important part of the sequence is the suffix (e.g. the question after the context, or the last response of the dialog).

## Your contribution

Perhaps I'll submit a PR, but it might take me some time as I'm close to some deadlines of mine :(

Contributor guide

Open the contributing guide

Research direction

Start by tracing the tokenizer truncation argument through the Transformers tokenizer entry points and existing truncation tests. Determine how prefix truncation should interact with max_length and the existing suffix-truncation behavior. Done means a supported option preserves the sequence suffix and tests cover the requested dialog and question-after-context use cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.