mlcommons / mlcommons/inference

Text to video benchmark failure

Open
#2,468 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.6k
Forks
650
Avg merge
1d 22h
Merged PRs (30d)
6

Description

I was currently testing the Text to Video benchmark and got the following error:

'/usr/bin/python3' run_mlperf.py             --model-path '/root/MLC/repos/local/cache/download-file_ml-model-wan2_925086cb/wan2-checkpoint/Wan2.2-T2V-A14B-Diffusers'             --dataset '/root/MLC/repos/local/cache/download-file_ml-dataset-text_bc68c0bf/text-to-video-dataset/vbench_prompts.txt'             --scenario Offline             --output-dir '/root/MLC/repos/local/cache/get-mlperf-inference-results-dir_d0b7b4d9/test_results/4e7cc549b15c-reference-gpu-pytorch-v2.5.1-cu126/wan-2.2-t2v-a14b/offline/performance/run_1'                             --user_conf '/root/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/2de1700dd92a4edfaecbbd022e0aa689.conf'   2>&1 | tee '/root/MLC/repos/local/cache/get-mlperf-inference-results-dir_d0b7b4d9/test_results/4e7cc549b15c-reference-gpu-pytorch-v2.5.1-cu126/wan-2.2-t2v-a14b/offline/performance/run_1/console.out'; echo ${PIPESTATUS[0]} > exitstatus
/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py:254: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  fixed_latent = torch.load(args.fixed_latent)
[Rank 0] 2026-01-27 17:14:29 - INFO - Loaded fixed latent from ./data/fixed_latent.pt with shape: torch.Size([1, 16, 21, 90, 160])
[Rank 0] 2026-01-27 17:14:29 - INFO - This latent will be reused for all generations
Loading checkpoint shards: 100%|██████████| 3/3 [00:00<00:00, 126.34it/s]
Loading checkpoint shards: 100%|██████████| 12/12 [00:06<00:00,  1.75it/s]
Loading checkpoint shards: 100%|██████████| 12/12 [00:15<00:00,  1.33s/it]it]
Loading pipeline components...: 100%|██████████| 6/6 [00:23<00:00,  3.94s/it]
[Rank 0] 2026-01-27 17:15:07 - INFO - Model loaded successfully!
100%|██████████| 20/20 [12:16<00:00, 36.81s/it]
Traceback (most recent call last):
  File "/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py", line 346, in <module>
    main()
  File "/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py", line 342, in main
    run_mlperf(args, config)
  File "/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py", line 327, in run_mlperf
    lg.StartTestWithLogSettings(
  File "/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py", line 100, in issue_queries
    "B", output.cpu().detach().numpy().tobytes()
AttributeError: 'numpy.ndarray' object has no attribute 'cpu'

Also, I see two files run_inference.py(which does not make use of loadgen) and run_mlperf.py(which makes use of loadgen) and the readme contains the instruction to run run_inference.py. Just wanted to make sure the instruction is pointed to the correct run file

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the benchmark using the command in the report, then inspect run_mlperf.py at line 100 where issue_queries handles the output and raises the AttributeError. Compare that path with run_inference.py and check the README quick-start instructions. Done means the loadgen run completes and the README points to the intended entry point.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.