anarchy-ai / anarchy-ai/LLM-VM

Fix Llama2 implementation: Cannot import name 'Llama2Tokenizer' from 'transformers'

Ouverte
#158 3 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
feat/enhancement
Langage dominant
Python
Étoiles
490
Forks
139
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

I created a quickstart test file which is mainly doing the same that appears on the documentation:
```python
from llm_vm.client import Client

client = Client(
big_model = 'neo',
big_model_config={'model_uri':'EleutherAI/gpt-neo-1.3B'},
small_model ='neo',
small_model_config={'model_uri':'EleutherAI/gpt-neo-125m'})

response=client.complete(prompt='What is Anarchy?', context='')
print(response)
```

But when I try to run it return it the next error:
```
python quickstart-test.py
Traceback (most recent call last):
File ".../LLM-VM/quickstart-test.py", line 2, in
from llm_vm.client import Client
File ".../LLM-VM/src/llm_vm/client.py", line 3, in
import llm_vm.onsite_llm as llms
File ".../LLM-VM/src/llm_vm/onsite_llm.py", line 6, in
from transformers import (
ImportError: cannot import name 'Llama2Tokenizer' from 'transformers' (.../LLM-VM/.venv/lib/python3.10/site-packages/transformers/__init__.py
```

If I checkout at commit 7a5877b6a everything works fine.

I checked the [huggingface transformers documentation and llama2 model is using the same transformer](https://huggingface.co/docs/transformers/model_doc/llama2) I also checked the [repo](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/tokenization_llama.py) and there is no llama2 folder nor Llama2Tokenizer, Llama2ForCausalLM classes.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.