microsoft / microsoft/LLMLingua
[Bug]: When calling slingua through PromptCompressor, use_llmlingua2 should also be set to true.
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 428
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 1
Description
Describe the bug
PromptCompressor(
model_name="SecurityLingua/securitylingua-xlm-s2s",
use_slingua=True,
)
on its own produces the error:
TypeError: XLMRobertaForTokenClassification.forward() got an unexpected keyword argument 'past_key_values'
.
The solution is to either call:
PromptCompressor(
model_name="SecurityLingua/securitylingua-xlm-s2s",
use_slingua=True,
use_llmlingua2=True,
)
or modify prompt_compressor.py to have
if self.use_llmlingua2 **or self.use_slingua:**
return self.compress_prompt_llmlingua2(
context,
rate=rate,
target_token=target_token,
use_context_level_filter=use_context_level_filter,
use_token_level_filter=use_token_level_filter,
target_context=target_context,
context_level_rate=context_level_rate,
context_level_target_token=context_level_target_token,
force_context_ids=force_context_ids,
return_word_label=return_word_label,
word_sep=word_sep,
label_sep=label_sep,
token_to_word=token_to_word,
force_tokens=force_tokens,
force_reserve_digit=force_reserve_digit,
drop_consecutive=drop_consecutive,
chunk_end_tokens=chunk_end_tokens,
)
Steps to reproduce
No response
Expected Behavior
No response
Logs
No response
Additional Information
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading prompt_compressor.py and trace how use_slingua and use_llmlingua2 select the compression path. Reproduce the shown PromptCompressor call, then verify that use_slingua=True routes through compress_prompt_llmlingua2 without requiring callers to set use_llmlingua2=True.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100