Cannot run the generation example notebook - which transformers version to use?

(Using Colab Free with T4)

Running `!pip install transformers ==4.10.3`
results in

`Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting transformers==4.10.3
Using cached transformers-4.10.3-py3-none-any.whl (2.8 MB)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.10.3) (3.12.0)
Collecting huggingface-hub>=0.0.12 (from transformers==4.10.3)
Using cached huggingface_hub-0.15.1-py3-none-any.whl (236 kB)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.10.3) (1.22.4)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from transformers==4.10.3) (23.1)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.10.3) (6.0)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.10.3) (2022.10.31)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.10.3) (2.27.1)
Collecting sacremoses (from transformers==4.10.3)
Using cached sacremoses-0.0.53-py3-none-any.whl
Collecting tokenizers<0.11,>=0.10.1 (from transformers==4.10.3)
Using cached tokenizers-0.10.3.tar.gz (212 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.10.3) (4.65.0)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.0.12->transformers==4.10.3) (2023.4.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.0.12->transformers==4.10.3) (4.5.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.10.3) (1.26.15)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.10.3) (2022.12.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.10.3) (2.0.12)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.10.3) (3.4)
Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from sacremoses->transformers==4.10.3) (1.16.0)
Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from sacremoses->transformers==4.10.3) (8.1.3)
Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from sacremoses->transformers==4.10.3) (1.2.0)
Building wheels for collected packages: tokenizers
error: subprocess-exited-with-error

× Building wheel for tokenizers (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Building wheel for tokenizers (pyproject.toml) ... error
ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

I looked in notebooks/mgpt_huggingface_generation_example.ipynb and saw that you used transformers==4.23.1. So, I tried the above notebook (from your README) with !pip install 4.23.1. This also did not run:

`model.cuda()
model.eval()
transformers.set_seed(1337)
for text in texts:
input_ids = tokenizer.encode(text, return_tensors="pt").cuda()
out = model.generate(
input_ids,
min_length=100,
max_length=100,
eos_token_id=5,
pad_token=1,
do_sample=True,
top_k=0,
top_p=0.9,
no_repeat_ngram_size=4)
generated_text = list(map(tokenizer.decode, out))[0]
print('---')
print(generated_text)`

results in

`---------------------------------------------------------------------------

ValueError Traceback (most recent call last)

[](https://localhost:8080/#) in ()
4 for text in texts:
5 input_ids = tokenizer.encode(text, return_tensors="pt").cuda()
----> 6 out = model.generate(
7 input_ids,
8 min_length=100,

2 frames

[/usr/local/lib/python3.10/dist-packages/transformers/generation_utils.py](https://localhost:8080/#) in _validate_model_kwargs(self, model_kwargs)
907
908 if unused_model_args:
--> 909 raise ValueError(
910 f"The following `model_kwargs` are not used by the model: {unused_model_args} (note: typos in the"
911 " generate arguments will also show up in this list)"

ValueError: The following `model_kwargs` are not used by the model: ['pad_token'] (note: typos in the generate arguments will also show up in this list)`
`
.

Which transformers version to use for generation?