huggingface / huggingface/transfer-learning-conv-ai
Difference in fine-tuned models linked in README and used in interact.py script
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 430
- PR merge metrics
- No merged PRs in 30d
Description
The pre-trained and fine-tuned model, given in the README of the repo under the 'Pretrained model' heading (linked [here](https://s3.amazonaws.com/models.huggingface.co/transfer-learning-chatbot/finetuned_chatbot_gpt.tar.gz)) is different from what gets used when we run `python interact.py`.
In the script utils.py, the link for the fine_tuned model has been given in the script as [https://s3.amazonaws.com/models.huggingface.co/transfer-learning-chatbot/gpt_personachat_cache.tar.gz](https://s3.amazonaws.com/models.huggingface.co/transfer-learning-chatbot/gpt_personachat_cache.tar.gz).
Are these two different models because when I tried to run the earlier fine-tuned model which was linked with the repo as a model-checkpoint with interact.py, following error was obtained -
Command used - `python3 ./interact.py --model_checkpoint='./finetuned_chatbot_gpt'`
Error:
Traceback (most recent call last):
File "./interact.py", line 155, in
run()
File "./interact.py", line 128, in run
model = model_class.from_pretrained(args.model_checkpoint)
File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_utils.py", line 558, in from_pretrained
model.__class__.__name__, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for OpenAIGPTLMHeadModel:
size mismatch for transformer.tokens_embed.weight: copying a param with shape torch.Size([40483, 768]) from checkpoint, the shape in current model is torch.Size([40478, 768]).
While just running `python3 interact.py`, the result was fine.
Kindly see to this issue
Contributor guide
No contributing guide indexed for this repository
Research direction
Compare the model URLs in the README and utils.py, then trace how interact.py passes --model_checkpoint to model_class.from_pretrained. Reproduce the reported vocabulary-size mismatch and determine which checkpoint and tokenizer configuration are compatible; done means the documented model and the default script use consistent, runnable artifacts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python, pytorch
- Domain
- documentation, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100