microsoft / microsoft/LLMLingua
PromptCompressor error - OpenAIGPTLMHeadModel.forward() got an unexpected keyword argument 'past_key_values'
@iofu728 is already working on this.
Since Jan 11, 2024.
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 428
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 1
Description
I am trying to use OpenAI GPT-2 model for prompt compression. However, getting error "OpenAIGPTLMHeadModel.forward() got an unexpected keyword argument 'past_key_values'". Has anyone faced/facing similar issue and how can this be fixed. Thanks.
`!pip install llmlingua
from llmlingua import PromptCompressor
llm_lingua = PromptCompressor(
model_name = "openai-gpt",
device_map="cpu",
)
prompt = """
The following is a conversation with an AI assistant. The assistant is helpful, creative, clever, and very friendly.
Human: Hello, who are you?
AI: I am an AI created by OpenAI. How can I help you today?
Human: I'd like to cancel my subscription.
AI: I'm sorry to hear that. What is your subscription number?
Human: 123456
AI: Thank you. Your subscription has been cancelled.
Human: Thank you. Goodbye!
AI: Goodbye!
"""
compressed_prompt = llm_lingua.compress_prompt(prompt, instruction="", question="", target_token=200)
`
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.