ml-explore / ml-explore/mlx-examples
fine-tuned T5 model fails to load after recent change
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
Directly importing a fine-tuned T5 model, like willwade/t5-small-spoken-typo fails with the following exception:
python t5.py --model willwade/t5-small-spoken-typo --prompt "grammar: A tasty apple"
File "./mlx-examples/t5/t5.py", line 495, in <module>
model, tokenizer = T5.from_pretrained(args.model, dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "./mlx-examples/t5/t5.py", line 427, in from_pretrained
model.load_weights(list(weights.items()))
File "/mlx/nn/layers/base.py", line 178, in load_weights
raise ValueError(f"Received parameters not in model: {extras}.")
ValueError: Received parameters not in model: decoder.layers.1.dense.wi.weight encoder.layers.0.dense.wi.weight decoder.layers.3.dense.wi.weight encoder.layers.1.dense.wi.weight encoder.layers.5.dense.wi.weight encoder.layers.3.dense.wi.weight encoder.layers.4.dense.wi.weight decoder.layers.0.dense.wi.weight decoder.layers.2.dense.wi.weight encoder.layers.2.dense.wi.weight decoder.layers.5.dense.wi.weight decoder.layers.4.dense.wi.weight.
After a recent commit https://github.com/ml-explore/mlx-examples/commit/d72fdeb4eeb0cf65f146ad3c3fbc48a2cd23974b, there is no longer a convert.py for T5 models.
Reverting https://github.com/ml-explore/mlx-examples/commit/d72fdeb4eeb0cf65f146ad3c3fbc48a2cd23974b and running convert.py to generate an .npz restores the expected functionality.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with t5/t5.py, especially T5.from_pretrained around line 427, and compare the loading behavior with commit d72fdeb4eeb0cf65f146ad3c3fbc48a2cd23974b. Reproduce the command using willwade/t5-small-spoken-typo and inspect the removed convert.py workflow; done means the fine-tuned model loads without unexpected dense.wi.weight parameters.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100