microsoft / microsoft/LLMLingua

LLMLingua and LongLLMLingua parameters question

Open
#76 3 comments 0 reactions 1 assignee View on GitHub

@iofu728 is already working on this.

Since Jan 25, 2024.

question
Dominant language
Python
Stars
6.7k
Forks
428
Avg merge
2d 4h
Merged PRs (30d)
1

Description

I read the issue #7, #12 and #49,
I guess the right papameters that LLMLingua uses are :
prompt = compressor.compress_prompt( context=xxx, instruction=xxx, question=xxx, ratio=0.75, iterative_size=100, context_budget="*2", ) ,
and LongLLMLingua is
prompt = compressor.compress_prompt( context=xxx, instruction=xxx, question=xxx, ratio=0.75, iterative_size=200, condition_compare=True, condition_in_question='after_condition', rank_method='longllmlingua', reorder_context='sort', dynamic_context_compression_ratio=0.3, context_budget="*2", )
I have some questions:

  1. In #7 , you said context_budget should be *1.3 or +300 in LongLLMLingua, and #12 ,you said context_budget should be +200 in LongLLMLingua, so I am confused by the setting of context_budget, and meanwhile, in the LLMLingua and LongLLMLingua papers, the context_budget seems to be *2 (in #49). So I want to know how to set context_budget in LLMLingua and LongLLMLingua.
  2. in #49, you said context_budget and token_budget_ratio can be considered part of the control coefficient parameter k. Can I think I just need to control the context_budget? Because in #7 and #12, you do not change the token_budget_ratio.
  3. What does dynamic_context_compression_ratio parameter correspond to in LongLLMLingua paper? I do not find it in the implementation details.
  4. The most important thing I want to know is whether the parameters I describes above are correct, and I want to get your LLMLingua and LongLLMLingua parameters you used actually in the paper, and i want to use TRUE parameters to run LLMLingua and LongLLMLingua experiments.

Please forgive my too long questions, because your LLMLingua and LongLLMLingua work are very interesting for me!
Looking forward to your reply.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.