microsoft / microsoft/LLMLingua
LLMLingua and LongLLMLingua parameters question
@iofu728 is already working on this.
Since Jan 25, 2024.
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 428
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 1
Description
I read the issue #7, #12 and #49,
I guess the right papameters that LLMLingua uses are :
prompt = compressor.compress_prompt(
context=xxx,
instruction=xxx,
question=xxx,
ratio=0.75,
iterative_size=100,
context_budget="*2",
) ,
and LongLLMLingua is
prompt = compressor.compress_prompt(
context=xxx,
instruction=xxx,
question=xxx,
ratio=0.75,
iterative_size=200,
condition_compare=True,
condition_in_question='after_condition',
rank_method='longllmlingua',
reorder_context='sort',
dynamic_context_compression_ratio=0.3,
context_budget="*2",
)
I have some questions:
- In #7 , you said
context_budgetshould be*1.3or+300in LongLLMLingua, and #12 ,you saidcontext_budgetshould be+200in LongLLMLingua, so I am confused by the setting ofcontext_budget, and meanwhile, in the LLMLingua and LongLLMLingua papers, thecontext_budgetseems to be*2(in #49). So I want to know how to setcontext_budgetin LLMLingua and LongLLMLingua. - in #49, you said
context_budgetandtoken_budget_ratiocan be considered part of the control coefficient parameterk. Can I think I just need to control thecontext_budget? Because in #7 and #12, you do not change thetoken_budget_ratio. - What does
dynamic_context_compression_ratioparameter correspond to in LongLLMLingua paper? I do not find it in the implementation details. - The most important thing I want to know is whether the parameters I describes above are correct, and I want to get your LLMLingua and LongLLMLingua parameters you used actually in the paper, and i want to use TRUE parameters to run LLMLingua and LongLLMLingua experiments.
Please forgive my too long questions, because your LLMLingua and LongLLMLingua work are very interesting for me!
Looking forward to your reply.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.