abertsch72 / abertsch72/unlimiformer
Script utilizing LLM
- Lenguaje dominante
- Python
- Estrellas
- 1.1k
- Forks
- 78
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
Can you provide a script similar to ``inference-example.py``, that utilises ``run_generation.py`` file? i.e instead of command like execution
``python src/run_generation.py --model_type llama --model_name_or_path meta-llama/Llama-2-13b-chat-hf \
--prefix "[INST] <>\n You are a helpful assistant. Answer with detailed responses according to the entire instruction or question. \n<>\n\n Summarize the following book: " \
--prompt example_inputs/harry_potter_full.txt \
--suffix " [/INST]" --test_unlimiformer --fp16 --length 200 --layer_begin 16 \
--index_devices 1 --datastore_device 1 ``
instead load the model and run inference from python script.
Thanks in advance!
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.