microsoft / microsoft/LLMLingua
keyError 'llama' when trying to running PromptCompressor()
@iofu728 is already working on this.
Since Jan 9, 2024.
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 428
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 1
Description
here is that stack trace. I can't for the life of me figure out what the source of this error is.
{
"name": "KeyError",
"message": "'llama'",
"stack": "---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
Cell In[2], line 4
1 # testing prompt compression using llmlingua
2 from llmlingua import PromptCompressor
----> 4 compressor = PromptCompressor(device_map="cpu")
5 compressed_prompt = compressor.compressed_prompt(prompt= text, instructions="identify the last unanswered question the trascript", question="",tartget_token =350)
File c:\Users\conrad.liburd\Anaconda3\envs\tapas\lib\site-packages\llmlingua\prompt_compressor.py:27, in PromptCompressor.init(self, model_name, device_map, model_config, open_api_config)
20 def init(
21 self,
22 model_name: str = "NousResearch/Llama-2-7b-hf",
(...)
25 open_api_config: dict = {},
26 ):
---> 27 self.load_model(model_name, device_map, model_config)
28 self.retrieval_model = None
29 self.retrieval_model_name = None
File c:\Users\conrad.liburd\Anaconda3\envs\tapas\lib\site-packages\llmlingua\prompt_compressor.py:40, in PromptCompressor.load_model(self, model_name, device_map, model_config)
38 if "trust_remote_code" not in model_config:
39 model_config["trust_remote_code"] = trust_remote_code
---> 40 config = AutoConfig.from_pretrained(
41 model_name, trust_remote_code=trust_remote_code
42 )
43 tokenizer = AutoTokenizer.from_pretrained(
44 model_name, trust_remote_code=trust_remote_code
45 )
46 if model_config.get("pad_to_left", True):
File c:\Users\conrad.liburd\Anaconda3\envs\tapas\lib\site-packages\transformers\models\auto\configuration_auto.py:917, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
915 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs)
916 elif "model_type" in config_dict:
--> 917 config_class = CONFIG_MAPPING[config_dict["model_type"]]
918 return config_class.from_dict(config_dict, **unused_kwargs)
919 else:
920 # Fallback: use pattern matching on the string.
921 # We go from longer names to shorter names to catch roberta before bert (for instance)
File c:\Users\conrad.liburd\Anaconda3\envs\tapas\lib\site-packages\transformers\models\auto\configuration_auto.py:623, in _LazyConfigMapping.getitem(self, key)
621 return self._extra_content[key]
622 if key not in self._mapping:
--> 623 raise KeyError(key)
624 value = self._mapping[key]
625 module_name = model_type_to_module_name(key)
KeyError: 'llama'"
}
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.