mlcommons / mlcommons/inference
ResNet50 inference command error
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 650
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 6
Description
I followed the document to inference ResNet50, using MLCommons-Python -> edge -> Tensorflow -> CUDA -> Native
The command is
cm run script --tags=run-mlperf,inference,_find-performance,_full,_r4.1 \
--model=resnet50 \
--implementation=reference \
--framework=tensorflow \
--category=edge \
--scenario=Offline \
--execution_mode=test \
--device=cuda \
--quiet \
--test_query_count=1000
And shows CM error: no scripts were found with above tags and variations, what's wrong?
The complete output is
INFO:root:* cm run script "run-mlperf inference _find-performance _full _r4.1"
INFO:root: * cm run script "detect os"
INFO:root: ! cd /home/eason/ml_workspace
INFO:root: ! call /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-os/run.sh from tmp-run.sh
INFO:root: ! call "postprocess" from /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-os/customize.py
INFO:root: * cm run script "detect cpu"
INFO:root: * cm run script "detect os"
INFO:root: ! cd /home/eason/ml_workspace
INFO:root: ! call /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-os/run.sh from tmp-run.sh
INFO:root: ! call "postprocess" from /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-os/customize.py
INFO:root: ! cd /home/eason/ml_workspace
INFO:root: ! call /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-cpu/run.sh from tmp-run.sh
INFO:root: ! call "postprocess" from /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-cpu/customize.py
INFO:root: * cm run script "get python3"
INFO:root: ! load /home/eason/CM/repos/local/cache/a41bcff19d784ed7/cm-cached-state.json
INFO:root:Path to Python: /home/eason/ml_workspace/py_venv/cm/bin/python3
INFO:root:Python version: 3.10.12
INFO:root: * cm run script "get mlcommons inference src"
INFO:root: ! load /home/eason/CM/repos/local/cache/c73bd23e67734e1b/cm-cached-state.json
INFO:root: * cm run script "get sut description"
INFO:root: * cm run script "detect os"
INFO:root: ! cd /home/eason/ml_workspace
INFO:root: ! call /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-os/run.sh from tmp-run.sh
INFO:root: ! call "postprocess" from /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-os/customize.py
INFO:root: * cm run script "detect cpu"
INFO:root: * cm run script "detect os"
INFO:root: ! cd /home/eason/ml_workspace
INFO:root: ! call /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-os/run.sh from tmp-run.sh
INFO:root: ! call "postprocess" from /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-os/customize.py
INFO:root: ! cd /home/eason/ml_workspace
INFO:root: ! call /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-cpu/run.sh from tmp-run.sh
INFO:root: ! call "postprocess" from /home/eason/CM/repos/mlcommons@cm4mlops/script/detect-cpu/customize.py
INFO:root: * cm run script "get python3"
INFO:root: ! load /home/eason/CM/repos/local/cache/a41bcff19d784ed7/cm-cached-state.json
INFO:root:Path to Python: /home/eason/ml_workspace/py_venv/cm/bin/python3
INFO:root:Python version: 3.10.12
INFO:root: * cm run script "get compiler"
INFO:root: ! load /home/eason/CM/repos/local/cache/0e347360381e4212/cm-cached-state.json
INFO:root: * cm run script "get cuda-devices"
INFO:root: * cm run script "get cuda _toolkit"
INFO:root: ! load /home/eason/CM/repos/local/cache/00ba83f0c4944122/cm-cached-state.json
INFO:root:ENV[CM_CUDA_PATH_LIB_CUDNN_EXISTS]: no
INFO:root:ENV[CM_CUDA_VERSION]: 12.5
INFO:root:ENV[CM_CUDA_VERSION_STRING]: cu125
INFO:root:ENV[CM_NVCC_BIN_WITH_PATH]: /usr/local/cuda/bin/nvcc
INFO:root:ENV[CUDA_HOME]: /usr/local/cuda
INFO:root: ! cd /home/eason/ml_workspace
INFO:root: ! call /home/eason/CM/repos/mlcommons@cm4mlops/script/get-cuda-devices/run.sh from tmp-run.sh
rm: cannot remove 'a.out': No such file or directory
Checking compiler version ...
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Jun__6_02:18:23_PDT_2024
Cuda compilation tools, release 12.5, V12.5.82
Build cuda_12.5.r12.5/compiler.34385749_0
Compiling program ...
Running program ...
/home/eason/ml_workspace
INFO:root: ! call "postprocess" from /home/eason/CM/repos/mlcommons@cm4mlops/script/get-cuda-devices/customize.py
GPU Device ID: 0
GPU Name: NVIDIA GeForce GTX 1070 Ti
GPU compute capability: 6.1
CUDA driver version: 12.2
CUDA runtime version: 12.5
Global memory: 8504934400
Max clock rate: 1683.000000 MHz
Total amount of shared memory per block: 49152
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block X: 1024
Max dimension size of a thread block Y: 1024
Max dimension size of a thread block Z: 64
Max dimension size of a grid size X: 2147483647
Max dimension size of a grid size Y: 65535
Max dimension size of a grid size Z: 65535
INFO:root: * cm run script "get generic-python-lib _package.dmiparser"
INFO:root: ! load /home/eason/CM/repos/local/cache/5fc4395af0ad4957/cm-cached-state.json
INFO:root: * cm run script "get cache dir _name.mlperf-inference-sut-descriptions"
INFO:root: ! load /home/eason/CM/repos/local/cache/071b923c360f4f19/cm-cached-state.json
Generating SUT description file for eason_Precision_Tower_5810-tensorflow
INFO:root: ! call "postprocess" from /home/eason/CM/repos/mlcommons@cm4mlops/script/get-mlperf-inference-sut-description/customize.py
INFO:root: * cm run script "get mlperf inference results dir"
INFO:root: ! load /home/eason/CM/repos/local/cache/3c30031394d145c4/cm-cached-state.json
INFO:root: * cm run script "install pip-package for-cmind-python _package.tabulate"
INFO:root: ! load /home/eason/CM/repos/local/cache/51d85c7579234247/cm-cached-state.json
INFO:root: * cm run script "get mlperf inference utils"
INFO:root: * cm run script "get mlperf inference src"
INFO:root: ! load /home/eason/CM/repos/local/cache/c73bd23e67734e1b/cm-cached-state.json
INFO:root: ! call "postprocess" from /home/eason/CM/repos/mlcommons@cm4mlops/script/get-mlperf-inference-utils/customize.py
Using MLCommons Inference source from /home/eason/CM/repos/local/cache/cade730fd2ef4531/inference
Running loadgen scenario: Offline and mode: performance
INFO:root:* cm run script "app mlperf inference generic _reference _resnet50 _tensorflow _cuda _test _r4.1_default _offline"
CM error: no scripts were found with above tags and variations
variation tags ['reference', 'resnet50', 'tensorflow', 'cuda', 'test', 'r4.1_default', 'offline'] are not matching for the found script app-mlperf-inference with variations dict_keys(['cpp', 'mil', 'mlcommons-cpp', 'ctuning-cpp-tflite', 'tflite-cpp', 'reference', 'python', 'nvidia', 'mlcommons-python', 'reference,gptj_', 'reference,sdxl_', 'reference,dlrm-v2_', 'reference,llama2-70b_', 'reference,mixtral-8x7b', 'reference,resnet50', 'reference,retinanet', 'reference,bert_', 'nvidia-original,r4.1-dev_default', 'nvidia-original,r4.1-dev_default,gptj_', 'nvidia-original,r4.1_default', 'nvidia-original,r4.1_default,gptj_', 'nvidia-original,r4.1-dev_default,llama2-70b_', 'nvidia-original,r4.1_default,llama2-70b_', 'nvidia-original', 'intel', 'intel-original', 'intel-original,gptj_', 'redhat', 'qualcomm', 'kilt', 'kilt,qaic,resnet50', 'kilt,qaic,retinanet', 'kilt,qaic,bert-99', 'kilt,qaic,bert-99.9', 'intel-original,resnet50', 'intel-original,retinanet', 'intel-original,bert-99', 'intel-original,bert-99.9', 'intel-original,gptj-99', 'intel-original,gptj-99.9', 'resnet50', 'retinanet', '3d-unet-99', '3d-unet-99.9', '3d-unet_', 'sdxl', 'llama2-70b_', 'llama2-70b-99', 'llama2-70b-99.9', 'mixtral-8x7b', 'rnnt', 'rnnt,reference', 'gptj-99', 'gptj-99.9', 'gptj', 'gptj_', 'bert_', 'bert-99', 'bert-99.9', 'dlrm_', 'dlrm-v2-99', 'dlrm-v2-99.9', 'dlrm_,nvidia', 'mobilenet', 'efficientnet', 'onnxruntime', 'tensorrt', 'tf', 'pytorch', 'openshift', 'ncnn', 'deepsparse', 'tflite', 'glow', 'tvm-onnx', 'tvm-pytorch', 'tvm-tflite', 'ray', 'cpu', 'cuda,reference', 'cuda', 'rocm', 'qaic', 'tpu', 'fast', 'test', 'valid,retinanet', 'valid', 'quantized', 'fp32', 'float32', 'float16', 'bfloat16', 'int4', 'int8', 'uint8', 'offline', 'multistream', 'singlestream', 'server', 'power', 'batch_size.#', 'r2.1_default', 'r3.0_default', 'r3.1_default', 'r4.0-dev_default', 'r4.0_default', 'r4.1-dev_default', 'r4.1_default'])
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the ResNet50 inference document and the reported cm run script command. Compare its requested variation tags with the app-mlperf-inference variations listed in the output, then inspect the matching MLCommons-Python entry point. Done means the documented command resolves to a script and starts the requested inference run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- cli, machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100