microsoft / microsoft/onnxruntime

numactl can't bind numa node in onnxruntime1.7

Open
#7,347 10 comments 0 reactions 1 assignee View on GitHub

@snnn is already working on this.

Since Jul 12, 2021.

feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 8h
Merged PRs (30d)
179

Description

Describe the bug
"numactl -m 0" can't bind the process to specific node or cores.

Urgency
since 1.7.0

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): centos7
  • ONNX Runtime installed from (source or binary): binary
  • ONNX Runtime version: 1.7.0
  • Python version: 3.7
  • Visual Studio version (if applicable):
  • GCC/Compiler version (if compiling from source):
  • CUDA/cuDNN version:
  • GPU model and memory:

To Reproduce

a tiny benchmark

import numpy as np
import onnxruntime
import onnx
from onnx import helper, TensorProto, numpy_helper, onnx_pb
import time

A = helper.make_tensor_value_info('A', TensorProto.FLOAT, [1, 3, 224, 224])
B = helper.make_tensor_value_info('B', TensorProto.FLOAT, [64, 3, 3, 3])
D = helper.make_tensor_value_info('D', TensorProto.FLOAT, [1, 64, 224, 224])
b_value = np.random.randn(64, 3, 3, 3).astype(np.float32)
B_init = helper.make_tensor('B', TensorProto.FLOAT, [64, 3, 3, 3],
                                b_value.reshape(1728).tolist())
conv_node = onnx.helper.make_node("Conv", ['A', 'B'], ['D'],
                                  name="conv",
                                  kernel_shape=[3, 3],
                                  pads=[1, 1, 1, 1])
graph = helper.make_graph([conv_node], 'test_graph_1', [A, B], [D], [B_init])
model = helper.make_model(graph)
onnx.save(model, "tiny_benchmark.onnx")

session = onnxruntime.InferenceSession("tiny_benchmark.onnx", None)
inputs = [np.random.random([1, 3, 224, 224]).astype(np.float32) for _ in range(1000)]
begin = time.time()
for input in inputs:
    _ = session.run(None, {"A": input})
print("use time {}".format(time.time()-begin))

and we want to use numactl

numactl -m 0 -C 0-3 python tiny_benchmark.py

Expected behavior
this will greatly affect performance of real model.
the process should be bond to node 0 core 0,1,2,3 as this image:
image

but on onnxruntime 1.7.0, the htop shows:
image

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.