Question: Can onnxruntime-genai.so be used in vanilla onnxruntime as a custom ops library?

Nobody has claimed this yet.

Hello,

I am using vanilla onnxruntime. When running a single loop cycle of a model such as Gemma 3, neither of CPU nor DNNL execution providers seem to do much (or even any) kernel fusion, thus performance is far from great.

I thought that GenAI might provide some LLM-specific operations and kernel fusions, so I registered onnxruntime-genai.so through RegisterCustomOpsLibrary_V2. I'm using the C API. The call returned a success status. So this part seems to be OK, it seems the library does export some custom operations, but I cant' be sure that any of this is recognized, since it seems that there's no way to inspect this in onnxruntime's api.

But then, when genai is registered as a custom ops provider, nothing changes. The log outputs are virtually the same, there seems to be no improvement with neither CPU ep nor DNNL ep, and the performance is virtually the same (poor).

Now, the main question is whether GenAI provides any specialized LLM stuff and whether it's supposed to be loaded as a custom ops library, or there's no effect because, although registered, it offers nothing in that domain, so nothing is used. If it is supposed to be used as a custom ops library, what might be the reason I'm not seeing any improvement? I'm using onnxruntime 1.24.1, and the latest onnxruntime-genai unpacked from the official tar.gz, on linux.

Start with the C API call to RegisterCustomOpsLibrary_V2 and compare the CPU and DNNL execution-provider logs described in the issue. Check how onnxruntime-genai is intended to integrate with onnxruntime 1.24.1 on Linux and whether its custom operations apply to the Gemma 3 model. Done means documenting whether this loading path is supported and explaining the unchanged behavior.

Written by the indexing model from the issue text.