[Bug]: Strange Perplexity Results

Describe the bug

Hi, I encountered an issue after updating to the latest code. Previously, the ppl results were normal, but after the update, I got abnormally high PPL values with minference or dense mode.

Any help or suggestions would be greatly appreciated!

Steps to reproduce

The following outputs were given when using minference when I ran the script experiments/ppl/run_ppl.sh with minference.

<---- MInference Config Detail ----> attn_type minference, kv_type dense
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████| 4/4 [00:29<00:00,  7.49s/it]
Starting test...
Testing with context length: 1000: 100%|███████████████████████████████████████████████| 1/1 [00:02<00:00,  2.21s/it]
Average PPL for 1K: 12910592.0
Testing with context length: 6210: 100%|███████████████████████████████████████████████| 1/1 [00:02<00:00,  2.21s/it]
Average PPL for 6K: 16580608.0
Testing with context length: 11420: 100%|██████████████████████████████████████████████| 1/1 [00:02<00:00,  2.83s/it]
Average PPL for 11K: 14680064.0
Testing with context length: 16630: 100%|██████████████████████████████████████████████| 1/1 [00:03<00:00,  3.69s/it]
Average PPL for 16K: 14680064.0
Testing with context length: 21840: 100%|██████████████████████████████████████████████| 1/1 [00:04<00:00,  4.41s/it]
Average PPL for 21K: 12910592.0
Testing with context length: 27050: 100%|██████████████████████████████████████████████| 1/1 [00:05<00:00,  5.15s/it]
Average PPL for 27K: 11403264.0
...[ignore else]

Expected Behavior

No response

Logs

No response

Additional Information

No response