Low MMLU score (86.63) on DeepSeek-R1-FP4 with trtllm-eval on B200, official score ~90.7

Hi,

We are running the DeepSeek-R1-FP4 model on a B200 GPU using the official trtllm-eval tool to evaluate MMLU performance. However, the accuracy I get is significantly lower than the official reported FP4 score.

Details:

• Model path: /data/models/nvidia/DeepSeek-R1-FP4
• Huggingface Model Card: https://huggingface.co/nvidia/DeepSeek-R1-FP4#evaluation
• Hardware: B200 GPU x8 NVLINK
• Command used:

cat >./extra-llm-api-config.yml <<EOF
use_cuda_graph: true
enable_attention_dp: true
EOF

trtllm-eval --model /data/models/nvidia/DeepSeek-R1-FP4 \
  --tp_size 8 \
  --kv_cache_free_gpu_memory_fraction 0.8 \
  --extra_llm_api_options ./extra-llm-api-config.yml \
  mmlu \
  --dataset_path /data/models/dataset/mmlu/mmlu-un-huggingface/data

• The MMLU dataset was downloaded from: https://people.eecs.berkeley.edu/~hendrycks/data.tar
• Obtained MMLU accuracy: 86.63
• Official FP4 MMLU accuracy reported: ~90.7
• We eval 3 times: 86.63 86.86 86.65

What I have verified:

• Using the latest official TensorRT-LLM docker image: nvcr.io/nvidia/tensorrt-llm/release:0.21.0rc0
• Following official example scripts for model loading and evaluation. https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/models/core/deepseek_v3/README.md#evaluation
• Dataset path and format are correct.

Questions:

• Is there an official recommended full startup script or configuration for running MMLU on DeepSeek-R1-FP4?
• Could the accuracy gap be due to missing or incorrect parameters such as number of shots, temperature, or evaluation subsets?
• Are there known compatibility issues between TensorRT-LLM versions and DeepSeek-R1-FP4 that might affect accuracy?
• Any suggestions on debugging or tuning to achieve official-level accuracy?

Start with the DeepSeek V3 evaluation section in examples/models/core/deepseek_v3/README.md and the supplied trtllm-eval command and extra-llm-api-config.yml. Reproduce the three reported MMLU runs using the stated model, dataset, release:0.21.0rc0 image, and TP8 configuration, then compare evaluation parameters with the official FP4 setup. Done means identifying the cause of the accuracy gap or documenting a confirmed compatibility or configuration issue.