huggingface / huggingface/course

Chapter 1.3 distilgpt2 example triggers a max_length/max_new_tokens conflict

Open Beginner friendly
#1,285 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
4.2k
Forks
1.4k
Avg merge
13m
Merged PRs (30d)
1

Description

## Description

The `distilgpt2` text-generation example in Chapter 1.3 uses the following code:

```python
from transformers import pipeline

generator = pipeline("text-generation", model="distilgpt2")
generator(
"In this course, we will teach you how to",
max_length=30,
num_return_sequences=2,
)
```

When I run this example with `transformers==4.57.0`, I receive the following warnings:

```text
Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy.

Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.

Both `max_new_tokens` (=256) and `max_length` (=30) seem to have been set. `max_new_tokens` will take precedence.
```

The warning reports that both `max_new_tokens=256` and the example's `max_length=30` are set. Since `max_new_tokens` takes precedence, the `max_length=30` argument shown in the course does not control the output length as intended.

I did not explicitly pass `max_new_tokens=256` in the example.

## Environment

* Operating system: Linux
* Python: 3.10.12
* `transformers`: 4.57.0
* Model: `distilgpt2`
* Pipeline task: `text-generation`

## Steps to reproduce

```python
from transformers import pipeline

generator = pipeline("text-generation", model="distilgpt2")

outputs = generator(
"In this course, we will teach you how to",
max_length=30,
num_return_sequences=2,
)

print(outputs)
```

## Expected behavior

The length parameter used in the course example should control the generated output without conflicting with another generation parameter.

## Actual behavior

The pipeline reports that `max_new_tokens=256` and `max_length=30` are both set. It then gives precedence to `max_new_tokens`, so the course example does not behave as described.

## Suggested update

The example could be updated to use `max_new_tokens` explicitly:

```python
from transformers import pipeline

generator = pipeline("text-generation", model="distilgpt2")
generator(
"In this course, we will teach you how to",
max_new_tokens=30,
num_return_sequences=2,
)
```

The surrounding explanation could also clarify the difference:

* `max_length` limits the total number of tokens, including the input tokens.
* `max_new_tokens` limits only the number of newly generated tokens.
* These parameters control token counts, not exact word counts.

This distinction is relevant because the exercise asks learners to generate sentences with a specified number of words, while the generation length parameters operate on tokens.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Chapter 1.3 distilgpt2 text-generation example and reproduce it with transformers==4.57.0. Review the surrounding explanation of max_length and max_new_tokens, then update the example so the intended length setting is unambiguous and verify that the conflicting-parameter warning is gone.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.