microsoft / microsoft/onnxruntime

Consecutive calls to Session::Run() with DML EP crashing

Open
#6,003 3 comments 0 reactions 1 assignee View on GitHub

@fdwr is already working on this.

Since Feb 12, 2021.

ep:DML
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

**Describe the bug**
I am trying to use DirectML EP for accelerating the network computations. The system loads the model and runs it multiple times, each time with a different input. When running on CPU, this works fine. When running on CUDA EP under Linux, this works fine. When running this on CUDA EP under Windows, there is a significant delay in the start of the processing, but eventually it runs. Using DML EP, the execution crashes on the second call to Session::Run().

This crash seems to depend from the model size. Using the example model below, the crash happens with kernel size of 512x512, but not with 384x384 (these numbers are probably valid only on my system). So a bigger model crashes. This would somehow suggest some memory handling issue.

I am using the dll of 1.5.2 available under releases.

**Urgency**
-

**System information**
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Win 10 Enterprise v1909
- ONNX Runtime installed from (source or binary): binary
- ONNX Runtime version: 1.5.2
- Python version: N/A
- Visual Studio version (if applicable): 16.8.2
- GCC/Compiler version (if compiling from source): N/A
- CUDA/cuDNN version: N/A
- GPU model and memory: GTX1660 Ti, 6GB

**To Reproduce**

Create model with pytorch (1.7):

```
class ConvAnaSyn(torch.nn.Module):
def __init__(self, n=384):
super(ConvAnaSyn, self).__init__()
self.len = n
self.hop = self.len // 2
self.weight = torch.nn.Parameter(torch.eye(self.len)[:, None, :], requires_grad=False)

def forward(self, x: torch.Tensor) -> torch.Tensor:
x = x.permute((0, 2, 1))
y = torch.nn.functional.conv1d(x, weight=self.weight, bias=None, stride=self.hop, padding=0)
z = torch.nn.functional.conv_transpose1d(y, weight=self.weight, bias=None, stride=self.hop, padding=0)
z = z.permute((0, 2, 1))
return z

n_dim = 384 # 512 to crash
model = ConvAnaSyn(n_dim).float()
x_in = torch.randn((3, 48000, 1), requires_grad=False)
x_out = model(x_in)
new_model_name = 'kernel_{}.onnx'.format(n_dim)

torch.onnx.export(model, (x_in, ), new_model_name, example_outputs=(x_out, ), input_names=['x_in'], output_names=['x_out'], opset_version=11, enable_onnx_checker=True)

```

C-program to run the model (linked against dll:

```
int main(int argc, const char* argv[]) {
//std::wstring model_file(L"kernel_384.onnx");
std::wstring model_file(L"kernel_512.onnx");
Ort::Env env(ORT_LOGGING_LEVEL_INFO, "test");
Ort::SessionOptions session_options;
session_options.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_ALL);
session_options.DisableMemPattern();
session_options.SetExecutionMode(ExecutionMode::ORT_SEQUENTIAL);
Ort::ThrowOnError(OrtSessionOptionsAppendExecutionProvider_DML(session_options, 0));
Ort::Session session(env, model_file.c_str(), session_options);
Ort::AllocatorWithDefaultOptions allocator;
std::vector input_node_dims = session.GetInputTypeInfo(0).GetTensorTypeAndShapeInfo().GetShape();
std::vector input_names = { "x_in" };
std::vector output_names = { "x_out" };
std::vector input_tensors;
size_t input_tensor_size = input_node_dims[0] * input_node_dims[1] * input_node_dims[2];
std::vector input_tensor_values(input_tensor_size);
auto memory_info = Ort::MemoryInfo::CreateCpu(OrtArenaAllocator, OrtMemTypeDefault);
input_tensors.emplace_back(Ort::Value::CreateTensor(memory_info, input_tensor_values.data(), input_tensor_size, input_node_dims.data(), input_node_dims.size()));

for (int loop_idx = 0; loop_idx < 10; loop_idx++) {
std::cout << "Loop number " << loop_idx + 1 << std::endl;
std::vector output_tensors = session.Run(Ort::RunOptions( nullptr ), input_names.data(), input_tensors.data(), input_names.size(), output_names.data(), output_names.size());
}
return EXIT_SUCCESS;
}
```

**Expected behavior**
Expecting multiple calls to Session::Run() to work without crashing also with DML.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Additional context**
Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.