huggingface / huggingface/course
Chapter 1.3 distilgpt2 example triggers a max_length/max_new_tokens conflict
- Dominant language
- MDX
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 13m
- Merged PRs (30d)
- 1
Description
## Description
The `distilgpt2` text-generation example in Chapter 1.3 uses the following code:
```python
from transformers import pipeline
generator = pipeline("text-generation", model="distilgpt2")
generator(
"In this course, we will teach you how to",
max_length=30,
num_return_sequences=2,
)
```
When I run this example with `transformers==4.57.0`, I receive the following warnings:
```text
Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy.
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
Both `max_new_tokens` (=256) and `max_length` (=30) seem to have been set. `max_new_tokens` will take precedence.
```
The warning reports that both `max_new_tokens=256` and the example's `max_length=30` are set. Since `max_new_tokens` takes precedence, the `max_length=30` argument shown in the course does not control the output length as intended.
I did not explicitly pass `max_new_tokens=256` in the example.
## Environment
* Operating system: Linux
* Python: 3.10.12
* `transformers`: 4.57.0
* Model: `distilgpt2`
* Pipeline task: `text-generation`
## Steps to reproduce
```python
from transformers import pipeline
generator = pipeline("text-generation", model="distilgpt2")
outputs = generator(
"In this course, we will teach you how to",
max_length=30,
num_return_sequences=2,
)
print(outputs)
```
## Expected behavior
The length parameter used in the course example should control the generated output without conflicting with another generation parameter.
## Actual behavior
The pipeline reports that `max_new_tokens=256` and `max_length=30` are both set. It then gives precedence to `max_new_tokens`, so the course example does not behave as described.
## Suggested update
The example could be updated to use `max_new_tokens` explicitly:
```python
from transformers import pipeline
generator = pipeline("text-generation", model="distilgpt2")
generator(
"In this course, we will teach you how to",
max_new_tokens=30,
num_return_sequences=2,
)
```
The surrounding explanation could also clarify the difference:
* `max_length` limits the total number of tokens, including the input tokens.
* `max_new_tokens` limits only the number of newly generated tokens.
* These parameters control token counts, not exact word counts.
This distinction is relevant because the exercise asks learners to generate sentences with a specified number of words, while the generation length parameters operate on tokens.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Chapter 1.3 distilgpt2 text-generation example and reproduce it with transformers==4.57.0. Review the surrounding explanation of max_length and max_new_tokens, then update the example so the intended length setting is unambiguous and verify that the conflicting-parameter warning is gone.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100