Failed to reproduce DeppSeek-R1-FP4 benchmarks

We are trying to reproduce DeepSeek-R1-FP4 benchmarks following the best practice: https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/Best_perf_practice_on_DeepSeek-R1_in_TensorRT-LLM.md

How did we build and run the container:

make -C docker build
make -C docker run DOCKER_RUN_ARGS="-v /workspace:/workspace"
python3 ./scripts/build_wheel.py --trt_root /usr/local/tensorrt --benchmarks --cuda_architectures "100-real"  --python_bindings --clean
pip install --user build/tensorrt_llm*.whl
export PATH=${HOME}/.local/bin:${PATH}
export PYTHONPATH=`pwd`

Then we followed the recipe to generate max throughput benchmark:

cat >/workspace/models/extra-llm-api-config.yml <<EOF
pytorch_backend_config:
    use_cuda_graph: true
    cuda_graph_padding_enabled: true
    cuda_graph_batch_sizes:
    - 1
    - 2
    - 4
    - 8
    - 16
    - 32
    - 64
    - 128
    - 256
    - 384
    print_iter_log: true
    enable_overlap_scheduler: true
enable_attention_dp: true
EOF

trtllm-bench -m nvidia/DeepSeek-R1-FP4 \
    --model_path /workspace/models/DeepSeek-R1-FP4 \
    throughput \
    --tp 8 \
    --ep 8 \
    --warmup 0 \
    --dataset /workspace/models/datasets/DeepSeek-R1-FP4/synthetic_1024_2048.txt \
    --backend pytorch \
    --max_batch_size 384 \
    --max_num_tokens 1536 \
    --num_requests 49152 \
    --concurrency 3072 \
    --kv_cache_free_gpu_mem_fraction 0.85 \
    --extra_llm_api_options /workspace/models/extra-llm-api-config.yml

Issue
The benchmark tool hanging at:

2025-04-23 18:42:08,745 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-04-23 18:42:08,746 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-04-23 18:42:08,746 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-04-23 18:42:08,747 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-04-23 18:42:08,747 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-04-23 18:42:08,747 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-04-23 18:42:08,747 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-04-23 18:42:08,748 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-04-23 18:42:08,758 - INFO - flashinfer.jit: Finished loading JIT ops: norm
2025-04-23 18:42:08,806 - INFO - flashinfer.jit: Finished loading JIT ops: norm
2025-04-23 18:42:08,856 - INFO - flashinfer.jit: Finished loading JIT ops: norm
2025-04-23 18:42:08,907 - INFO - flashinfer.jit: Finished loading JIT ops: norm
2025-04-23 18:42:08,957 - INFO - flashinfer.jit: Finished loading JIT ops: norm
2025-04-23 18:42:09,009 - INFO - flashinfer.jit: Finished loading JIT ops: norm
2025-04-23 18:42:09,059 - INFO - flashinfer.jit: Finished loading JIT ops: norm
2025-04-23 18:42:09,109 - INFO - flashinfer.jit: Finished loading JIT ops: norm
2025-04-23 18:42:09,713 - INFO - flashinfer.jit: Loading JIT ops: silu_and_mul
2025-04-23 18:42:09,739 - INFO - flashinfer.jit: Loading JIT ops: silu_and_mul
2025-04-23 18:42:09,806 - INFO - flashinfer.jit: Loading JIT ops: silu_and_mul
2025-04-23 18:42:09,870 - INFO - flashinfer.jit: Loading JIT ops: silu_and_mul
2025-04-23 18:42:09,915 - INFO - flashinfer.jit: Loading JIT ops: silu_and_mul
2025-04-23 18:42:09,940 - INFO - flashinfer.jit: Loading JIT ops: silu_and_mul
2025-04-23 18:42:09,995 - INFO - flashinfer.jit: Loading JIT ops: silu_and_mul
2025-04-23 18:42:10,038 - INFO - flashinfer.jit: Loading JIT ops: silu_and_mul

Did I miss any setup step that will cause this issue? Thanks!

Start with the linked DeepSeek-R1 TensorRT-LLM best-practice document, then compare the reported docker commands, scripts/build_wheel.py invocation, extra-llm-api-config.yml, and trtllm-bench command with the hang logs. Reproduce the benchmark if possible and identify the missing setup step or condition that prevents it from progressing past FlashInfer JIT loading.