[Question]: Reproduce LongLLMLingua on the LongBench MultiDoc dataset

Describe the issue

We are interested in your longllmlingua results on longbench. We referred to these two parts of your code[https://github.com/microsoft/LLMLingua/blob/main/experiments/llmlingua2/evaluation/compress.py
][https://github.com/microsoft/LLMLingua/blob/main/experiments/llmlingua2/evaluation/eval_longbench.py], and other settings are as follows.

target token: 3000
compresse model: Llama-2-7b-chat-hf
llm model: Mistral-7B-v0.1
compress config:

contexts_list = origin.split("\n")
demonstration = [
    "\n".join(contexts_list[ii : ii + 4]) for ii in range(0, len(contexts_list), 4)
]
comp_dict = compressor.compress_prompt(
    demonstration,
    instruction=dataset2instruction[copy.deepcopy(sample["task"])],
    question=dataset2question[copy.deepcopy(sample["task"])].format(input=copy.deepcopy(sample["question"])),
    target_token=args.target_token,
    condition_compare=True,
    condition_in_question="after",
    rank_method="longllmlingua",
    use_sentence_level_filter=False,
    context_budget="+100",
    dynamic_context_compression_ratio=0.4,
    reorder_context="sort",
)

The results when running the local model (Mistral-7B-v0.1) appear different from the conclusions in the paper:

1. The overall latency increases directly because it is a two-step reasoning overhead;
2. The accuracy problem is very serious.
3. Bugs occurs on the musique dataset.

result:
task: 2wikimqa
compress_time:816.7587068080902 s
Original Prompt: {'2wikimqa': {'score': 34.85, 'num': 200}, 'avg': 34.85} predict_time:304.49882340431213 s
LongLLMLingua: {'2wikimqa': {'score': 34.99, 'num': 200}, 'avg': 34.99} predict_time:224.4709668159485 s

task: dureader
compress_time:986.9709990024567 s
Original Prompt: {'dureader': {'score': 0.68, 'num': 200}, 'avg': 0.68} predict_time:888.872100353241 s
LongLLMLingua: {'dureader': {'score': 0.02, 'num': 200}, 'avg': 0.02} predict_time:735.0324370861053 s

task: hotpotqa
compress_time:1154.135990858078 s
Original Prompt: {'hotpotqa': {'score': 41.16, 'num': 200}, 'avg': 41.16} predict_time:342.1117823123932 s
LongLLMLingua: {'hotpotqa': {'score': 38.77, 'num': 200}, 'avg': 38.77} predict_time:237.45726418495178 s

task: musique

Traceback (most recent call last):
  File "/data2/lkk/yuxiang/llmlingua/compress.py", line 174, in <module>
    comp_dict = compressor.compress_prompt(
  File "/home/lvkaokao/anaconda3/envs/llmlingua1/lib/python3.10/site-packages/llmlingua/prompt_compressor.py", line 676, in compress_prompt
    start = self.get_prefix_length(prefix + "\n\n", context[0])
  File "/home/lvkaokao/anaconda3/envs/llmlingua1/lib/python3.10/site-packages/llmlingua/prompt_compressor.py", line 995, in get_prefix_length
    assert self.tokenizer.decode(full_input_ids[i:]) == text[:100]
AssertionError

Test hardware platform:
We used 1 Nvidia A100-SXM4-80GB per experiment.

I would like to communicate with you the reason for these phenomenon. Thank you very much for your reply.