Automated command for llama2-70b: Changing Batch Size fails

Hello mlcommons team,

I want to run the "Automated command to run the benchmark via MLCommons CM" (from the example: https://github.com/mlcommons/inference/tree/master/language/llama2-70b) with a different batch size, but I am getting the following error:

Run Directory: /root/CM/repos/local/cache/19f3466c31404fb9/inference/language/llama2-70b

CMD: /root/CM/repos/local/cache/9798222eb1384e65/mlperf/bin/python3 main.py  --scenario Offline --dataset-path /root/CM/repos/local/cache/cf3f035c15414140/open_orca/open_orca_gpt4_tokenized_llama.sampled_24576.pkl.gz --device cuda:0   --max-batchsize 100 --batch-size 8 --mlperf-conf '/root/CM/repos/local/cache/19f3466c31404fb9/inference/mlperf.conf' --user-conf '/root/CM/repos/mlcommons@cm4mlops/script/generate-mlperf-inference-user-conf/tmp/7fd03acd2fb849d3ad4d7b62d8bbbed1.conf' --output-log-dir /root/CM/repos/local/cache/e905aba5cd2047cf/test_results/969a89282f1c-reference-gpu-pytorch-v2.4.0-default_config/llama2-70b-99/offline/performance/run_1 --dtype float16 --model-path /Llama-2-70b-chat-hf 2>&1 ; echo \$? > exitstatus | tee '/root/CM/repos/local/cache/e905aba5cd2047cf/test_results/969a89282f1c-reference-gpu-pytorch-v2.4.0-default_config/llama2-70b-99/offline/performance/run_1/console.out'

INFO:root:         ! cd /root/CM/repos/local/cache/05e8741e40c349bf
INFO:root:         ! call /root/CM/repos/mlcommons@cm4mlops/script/benchmark-program/run-ubuntu.sh from tmp-run.sh
/root/CM/repos/local/cache/9798222eb1384e65/mlperf/bin/python3 main.py  --scenario Offline --dataset-path /root/CM/repos/local/cache/cf3f035c15414140/open_orca/open_orca_gpt4_tokenized_llama.sampled_24576.pkl.gz --device cuda:0   --max-batchsize 100 --batch-size 8 --mlperf-conf '/root/CM/repos/local/cache/19f3466c31404fb9/inference/mlperf.conf' --user-conf '/root/CM/repos/mlcommons@cm4mlops/script/generate-mlperf-inference-user-conf/tmp/7fd03acd2fb849d3ad4d7b62d8bbbed1.conf' --output-log-dir /root/CM/repos/local/cache/e905aba5cd2047cf/test_results/969a89282f1c-reference-gpu-pytorch-v2.4.0-default_config/llama2-70b-99/offline/performance/run_1 --dtype float16 --model-path /Llama-2-70b-chat-hf 2>&1 ; echo $? > exitstatus | tee '/root/CM/repos/local/cache/e905aba5cd2047cf/test_results/969a89282f1c-reference-gpu-pytorch-v2.4.0-default_config/llama2-70b-99/offline/performance/run_1/console.out'
usage: main.py [-h] [--scenario {Offline,Server}] [--model-path MODEL_PATH] [--dataset-path DATASET_PATH] [--accuracy] [--dtype DTYPE] [--device {cpu,cuda:0}] [--audit-conf AUDIT_CONF] [--mlperf-conf MLPERF_CONF]
               [--user-conf USER_CONF] [--total-sample-count TOTAL_SAMPLE_COUNT] [--batch-size BATCH_SIZE] [--output-log-dir OUTPUT_LOG_DIR] [--enable-log-trace] [--num-workers NUM_WORKERS]
main.py: error: unrecognized arguments: --max-batchsize 100
./run.sh: line 56: 2: command not found

CM error: Portable CM script failed (name = benchmark-program, return code = 256)

I seems that it sets the max-batchsize even tho I specified the --batch-size=100

I am running the following command:

cm run script --tags=run-mlperf,_full,inference,_r4.1 \
   --model=llama2-70b-99 \
   --implementation=reference \
   --framework=pytorch \
   --category=datacenter \
   --scenario=Offline \
   --device=cuda  \
   --quiet \
   --test_query_count=1000 --batch-size=100 --execution_mode=test \
   --precision=float16 --env.LLAMA2_CHECKPOINT_PATH=/Llama-2-70b-chat-hf

Start with the llama2-70b example under language/llama2-70b and trace how the CM command builds the invocation of main.py; the reported failure includes an unrecognized --max-batchsize argument and points to run.sh line 56. Reproduce the supplied command and consider the issue done when changing the batch-size no longer adds an unsupported argument and the benchmark runs.