microsoft / microsoft/onnxruntime
numactl can't bind numa node in onnxruntime1.7
@snnn is already working on this.
Since Jul 12, 2021.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 179
Description
Describe the bug
"numactl -m 0" can't bind the process to specific node or cores.
Urgency
since 1.7.0
System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): centos7
- ONNX Runtime installed from (source or binary): binary
- ONNX Runtime version: 1.7.0
- Python version: 3.7
- Visual Studio version (if applicable):
- GCC/Compiler version (if compiling from source):
- CUDA/cuDNN version:
- GPU model and memory:
To Reproduce
a tiny benchmark
import numpy as np
import onnxruntime
import onnx
from onnx import helper, TensorProto, numpy_helper, onnx_pb
import time
A = helper.make_tensor_value_info('A', TensorProto.FLOAT, [1, 3, 224, 224])
B = helper.make_tensor_value_info('B', TensorProto.FLOAT, [64, 3, 3, 3])
D = helper.make_tensor_value_info('D', TensorProto.FLOAT, [1, 64, 224, 224])
b_value = np.random.randn(64, 3, 3, 3).astype(np.float32)
B_init = helper.make_tensor('B', TensorProto.FLOAT, [64, 3, 3, 3],
b_value.reshape(1728).tolist())
conv_node = onnx.helper.make_node("Conv", ['A', 'B'], ['D'],
name="conv",
kernel_shape=[3, 3],
pads=[1, 1, 1, 1])
graph = helper.make_graph([conv_node], 'test_graph_1', [A, B], [D], [B_init])
model = helper.make_model(graph)
onnx.save(model, "tiny_benchmark.onnx")
session = onnxruntime.InferenceSession("tiny_benchmark.onnx", None)
inputs = [np.random.random([1, 3, 224, 224]).astype(np.float32) for _ in range(1000)]
begin = time.time()
for input in inputs:
_ = session.run(None, {"A": input})
print("use time {}".format(time.time()-begin))
and we want to use numactl
numactl -m 0 -C 0-3 python tiny_benchmark.py
Expected behavior
this will greatly affect performance of real model.
the process should be bond to node 0 core 0,1,2,3 as this image:

but on onnxruntime 1.7.0, the htop shows:

Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.