Dependency Build Failures & Argument Conflicts when Benchmarking DeepSeek models (NGC 24.10 / 25.04)

**Environment SetUp**
Codebase: Megatron-LM (Main branch) + Megatron-MoE-ModelZoo

Docker Images Tested: nvcr.io/nvidia/pytorch:24.10-py3 and 25.04-py3

Goal: Run DeepSeek-V2-Lite benchmark using interactive_benchmarking.sh with --mock-data.
**Describe the bug**

I encountered significant difficulties setting up the benchmarking environment due to dependency version mismatches (nv-grouped-gemm) and script logic conflicts. Below are the details for different container versions.

**Steps/Code to reproduce bug**

1. Issues with NGC 24.10 (Stable) - nv-grouped-gemm Build Failure
In the standard 24.10 container, pip install . for Megatron-LM triggers the build of the latest nv-grouped-gemm (v1.x). This compilation fails due to CUTLASS syntax incompatibility with the container's CUDA compiler.

Error Log:

/tmp/pip-install.../csrc/permute.cu(113): error: too many arguments for class template "cutlass::arch::global_load"
ninja: build stopped: subcommand failed.
ERROR: Failed building wheel for nv-grouped-gemm

Issues with NGC 25.04 (Preview) - PyTorch Inductor/Triton Crash
In the newer 25.04 container, the build succeeds, but the runtime crashes immediately during JIT warmup due to a PyTorch Nightly vs Triton incompatibility.

Error Log:

torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
ImportError: cannot import name 'triton_key' from 'triton.compiler.compiler'
Workaround: Using --no-bias-dropout-fusion and --no-bias-gelu-fusion flags, or commenting out _warmup_jit_function() in initialize.py.

3. Argument Conflict in Benchmarking Script (All Versions)
When running interactive_benchmarking.sh with --mock-data, the script logic in common.conf (sourced by the main script) automatically appends a --data-path argument based on the config file, causing a conflict in Megatron-LM.

Command:

MODEL=DeepSeek-V2-Lite ... bash interactive_benchmarking.sh --mock-data ...
Error:

AssertionError: A single data source must be provided in training mode, else None
(The script passes both --data-path ... and --mock-data to pretrain_gpt.py)