[Question]: Reproducing the score of official microsoft/llmlingua-2-xlm-roberta-large-meetingbank

Describe the issue

Following the issue 155, I'm trying to reproduce the results of the official llmlingua-2-xlm-roberta-large-meetingbank model using Mistral-7B as black-box llm.

In specific, I tried to fine-tune the XLM-RoBERTa model with the officially provided dataset, using this train.sh.

Here is my detailed process:

1. Format, label, and filter the official dataset with reference to the collect_data.sh.
2. Fine-tune the XLM-RoBERTa model using train.sh, with hyperparameters from the LLMLingua-2 paper.

Here are the current issues:

1. It's hard to reproduce the Table-4 results of LLMLingua-2 paper, or even scores in issue 155. Here are my reproduced results:

2. I found that the official llmlingua-2-xlm-roberta-large-meetingbank model weight has the word_embedding size of [250102, 1024]. This is larger than the original [250002, 1024] size of XLM-RoBERTa.
   I guess this is relevant to the added special tokens in prompt_compressor.py, but train_roberta.py example does nothing about this, so my fine-tuned model has the same word_embedding weight size with the original RoBERTa ([250002, 1024]).

• I tried to resize token embedding size first then fine-tune, but the results were almost the same.

3. I guess the example in train.sh doesn't use filtered results, which is named annotation_kept_cs512_meetingbank_train_formated.pt in collect_data.sh. This seems like a minor issue :)

If the process of training the official model is the same as the process provided as an example here, can you please let me know what needs to be changed in the above process?
Thank you for reading.