[Question] Unable to reproduce LLaMA2-70B benchmark results - Missing TensorRT-LLM v1.1.0rc6 and configuration details

Hi Azure team,

We're attempting to reproduce the MLPerf Inference v5.1 LLaMA2-70B benchmark results on Azure ND GB300 v6 as documented in [this blog post](https://techcommunity.microsoft.com/blog/azurehighperformancecomputingblog/breaking-the-million-token-barrier-the-technical-achievement-of-azure-nd-gb300-v/4466080) and the [1M_ND_GB300_v6_Inference](https://github.com/Azure/AI-benchmarking-guide/tree/main/Azure_Results/1M_ND_GB300_v6_Inference) directory.

However, we're encountering several issues that prevent us from reproducing your results:

## Issues

### 1. TensorRT-LLM Version Mismatch

The documentation specifies installing `tensorrt_llm-1.1.0rc6-cp312-cp312-linux_aarch64.whl`:

```bash
pip install build/tensorrt_llm-1.1.0rc6-cp312-cp312-linux_aarch64.whl
```

__Problem:__ The git tag `v1.1.0rc6` does not exist in the [NVIDIA/TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) repository. Available tags only go up to `v1.1.0rc5`:

```javascript
v1.1.0rc0
v1.1.0rc1
v1.1.0rc2
v1.1.0rc2.post1
v1.1.0rc2.post2
v1.1.0rc3
v1.1.0rc4
v1.1.0rc5
```

__Questions:__

- Was v1.1.0rc6 an internal Azure build or pre-release version?
- What is the exact commit hash or source used to build this version?
- Can you provide the exact TensorRT-LLM source or wheel file used for benchmarking?

### 2. Missing Configuration Details

The README provides high-level steps, but we need more specific configuration details:

__TensorRT-LLM Configuration:__

- What are the exact engine build parameters used?
- Batch sizes, sequence lengths, and other model compilation settings?
- Any custom patches or modifications to TensorRT-LLM?

__Loadgen Configuration:__

- Specific QPS targets and performance targets used
- Dataset preparation specifics
- Any custom loadgen parameters

__Environment Variables:__

- Are there specific CUDA environment variables that need to be set?
- Memory/cache configurations?
- Any system-level optimizations or tuning?

### 3. Missing Files/Configurations

Could you please share:

1. The complete TensorRT-LLM engine build configuration files
2. The loadgen configuration files used for the benchmark
3. Any environment setup scripts or configuration files
4. The exact Python package versions (full `pip freeze` output) used in the environment

## Request

To help the community reproduce your excellent results, could you please provide:

1. ✅ The exact TensorRT-LLM version/commit used (or the wheel file itself)
2. ✅ Complete configuration files for TensorRT-LLM engine building
3. ✅ Complete loadgen configuration files
4. ✅ List of environment variables and system configurations
5. ✅ Full Python package dependency list (`pip freeze` output)
6. ✅ Any custom patches or modifications made to TensorRT-LLM or other components

This information would be invaluable for researchers and engineers trying to reproduce and build upon your work.

Thank you for your time and for sharing this impressive benchmark!

---

__References:__

- Blog post: [](https://techcommunity.microsoft.com/blog/azurehighperformancecomputingblog/breaking-the-million-token-barrier-the-technical-achievement-of-azure-nd-gb300-v/4466080)
- Repository: [](https://github.com/Azure/AI-benchmarking-guide/tree/main/Azure_Results/1M_ND_GB300_v6_Inference)