mlcommons / mlcommons/inference
Text to video benchmark failure
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 650
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 6
Description
I was currently testing the Text to Video benchmark and got the following error:
'/usr/bin/python3' run_mlperf.py --model-path '/root/MLC/repos/local/cache/download-file_ml-model-wan2_925086cb/wan2-checkpoint/Wan2.2-T2V-A14B-Diffusers' --dataset '/root/MLC/repos/local/cache/download-file_ml-dataset-text_bc68c0bf/text-to-video-dataset/vbench_prompts.txt' --scenario Offline --output-dir '/root/MLC/repos/local/cache/get-mlperf-inference-results-dir_d0b7b4d9/test_results/4e7cc549b15c-reference-gpu-pytorch-v2.5.1-cu126/wan-2.2-t2v-a14b/offline/performance/run_1' --user_conf '/root/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/2de1700dd92a4edfaecbbd022e0aa689.conf' 2>&1 | tee '/root/MLC/repos/local/cache/get-mlperf-inference-results-dir_d0b7b4d9/test_results/4e7cc549b15c-reference-gpu-pytorch-v2.5.1-cu126/wan-2.2-t2v-a14b/offline/performance/run_1/console.out'; echo ${PIPESTATUS[0]} > exitstatus
/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py:254: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
fixed_latent = torch.load(args.fixed_latent)
[Rank 0] 2026-01-27 17:14:29 - INFO - Loaded fixed latent from ./data/fixed_latent.pt with shape: torch.Size([1, 16, 21, 90, 160])
[Rank 0] 2026-01-27 17:14:29 - INFO - This latent will be reused for all generations
Loading checkpoint shards: 100%|██████████| 3/3 [00:00<00:00, 126.34it/s]
Loading checkpoint shards: 100%|██████████| 12/12 [00:06<00:00, 1.75it/s]
Loading checkpoint shards: 100%|██████████| 12/12 [00:15<00:00, 1.33s/it]it]
Loading pipeline components...: 100%|██████████| 6/6 [00:23<00:00, 3.94s/it]
[Rank 0] 2026-01-27 17:15:07 - INFO - Model loaded successfully!
100%|██████████| 20/20 [12:16<00:00, 36.81s/it]
Traceback (most recent call last):
File "/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py", line 346, in <module>
main()
File "/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py", line 342, in main
run_mlperf(args, config)
File "/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py", line 327, in run_mlperf
lg.StartTestWithLogSettings(
File "/root/MLC/repos/local/cache/get-git-repo_inference-src_6efa07d9/inference/text_to_video/wan2.2-t2v-14b/run_mlperf.py", line 100, in issue_queries
"B", output.cpu().detach().numpy().tobytes()
AttributeError: 'numpy.ndarray' object has no attribute 'cpu'
Also, I see two files run_inference.py(which does not make use of loadgen) and run_mlperf.py(which makes use of loadgen) and the readme contains the instruction to run run_inference.py. Just wanted to make sure the instruction is pointed to the correct run file
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the benchmark using the command in the report, then inspect run_mlperf.py at line 100 where issue_queries handles the output and raises the AttributeError. Compare that path with run_inference.py and check the README quick-start instructions. Done means the loadgen run completes and the README points to the intended entry point.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100