microsoft / microsoft/onnxruntime

Cannot register custom DML operator

Open
#16,347 5 comments 0 reactions 1 assignee View on GitHub

@RandySheriffH is already working on this.

Since Jun 26, 2023.

ep:DML platform:windows
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Hello,

I'm attempting to register a custom operator for the DmlExectionProvider, and getting a crash from `GraphPartitioner.cpp`.

The message is: `Assertion failed: createInfo != nullptr, file \core\providers\dml\DmlExecutionProvider\src\GraphPartitioner.cpp, line 166`.

```c++
struct MyCustopOpKernel { ... };
struct MyCustomOp : Ort::CustomOpBase { ... }

Ort::CustomOpDomain custom_op_domain("my_domain");
MyCustomOp custom_op{};
custom_op_domain.Add(&custom_op);

Ort::SessionOptions sessionOptions;
sessionOptions.Add(custom_op_domain);
// ... other DML-specific session options configurations..
ortDmlApi->SessionOptionsAppendExecutionProvider_DML1(sessionOptions, dmlDevice, dmlQueue);
// crash occurs on this line:
ortSession = std::make_unique(ortEnv, modelData.data(), modelData.size(), sessionOptions);
```

I know that onnxruntime is able to recognize my op declaration, because if the domain name or the op name don't match what's in the model, I get a helpful message letting me know that the op/function was not found. I when correctly configured, it gets past that point, only to crash deeper in the Ort::Session initialization. Based on the where the crash happens in the ort source code, it seems like I need to access the `RegisterDmlOperators` function from `DmlExecutionProvider.h` file. But its not clear that this is part of the public API? It's not exported from `dml_provider_factory.h`, which I had been using as my entry point to the onnxruntime DirectML functionality. So I'm wondering if implementing custom DML operator is even supported? And if so, what am I missing here?

### To reproduce

I am happy to upload a more complete repro but I wanted to get a confirmation that what I'm trying to do is even supported first :)

### Urgency

:)

### Platform

Windows

### OS Version

10

### ONNX Runtime Installation

Built from Source

### ONNX Runtime Version or Commit ID

latest (1.15)

### ONNX Runtime API

C++

### Architecture

X64

### Execution Provider

DirectML

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.