microsoft / microsoft/onnxruntime

[WebGPU Plugin EP] GetSharedAllocator returns null after registration despite exposing DEFAULT MemoryInfo

Open
#32,164 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ep:WebGPU platform:web stale
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

After registering the official WebGPU Plugin EP, `OrtEpDevice::GetMemoryInfo(OrtDeviceMemoryType_DEFAULT)` returns a valid GPU `OrtMemoryInfo`, but `OrtEnv::GetSharedAllocator()` returns `nullptr` for that memory info.

This appears inconsistent with the `GetSharedAllocator` C API documentation:

> By default there is a shared allocator created for all OrtEpDevice instances, so if you get the OrtMemoryInfo from the OrtEpDevice using EpDevice_MemoryInfo a shared allocator is guaranteed to exist.

Calling `CreateSharedAllocator()` explicitly for the same device and memory type succeeds. A subsequent `GetSharedAllocator()` returns that allocator, and allocation, CPU/GPU copies, session creation, and WebGPU inference all succeed.

### Environment

- Windows x64
- ONNX Runtime core: `1.29.0`
- ORT commit from the official SDK: `2e2543fbe9fae542f921d47a72d21d5a4ef0b710`
- WebGPU Plugin EP: official PyPI package `onnxruntime-ep-webgpu==0.2.1`
- Plugin library: `onnxruntime_providers_webgpu.dll`

### Minimal reproduction

```cpp
#include

#include
#include

int main() {
Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "webgpu-shared-allocator");
env.RegisterExecutionProviderLibrary(
"webgpu_plugin_test",
ORT_TSTR("onnxruntime_providers_webgpu.dll"));

Ort::ConstEpDevice webgpu_device;
for (const auto& device : env.GetEpDevices()) {
if (std::strcmp(device.EpName(), "WebGpuExecutionProvider") == 0) {
webgpu_device = device;
break;
}
}
assert(static_cast(webgpu_device) != nullptr);

const auto memory_info =
webgpu_device.GetMemoryInfo(OrtDeviceMemoryType_DEFAULT);
assert(static_cast(memory_info) != nullptr);

// Unexpected: returns nullptr.
auto allocator = env.GetSharedAllocator(memory_info);
assert(static_cast(allocator) != nullptr);
}
```

### Actual behavior

```text
allocator_name=WebGPU_Buf
allocator_type=device (0)
device_type=GPU (1)
vendor_id=0
device_id=0
memory_type=default (0)
device_memory_type=device (0)
GetSharedAllocator returned null before CreateSharedAllocator.
```

### Expected behavior

Because the WebGPU `OrtEpDevice` exposes a non-null `DEFAULT` `OrtMemoryInfo`, `GetSharedAllocator(memory_info)` should return the default shared allocator immediately after `RegisterExecutionProviderLibrary()`.

### Additional verification

The following works:

```cpp
auto created = env.CreateSharedAllocator(
webgpu_device,
OrtDeviceMemoryType_DEFAULT,
OrtDeviceAllocator,
nullptr);

auto retrieved = env.GetSharedAllocator(memory_info);
assert(static_cast(retrieved) ==
static_cast(created));
```

A 4 MiB WebGPU tensor can then be allocated before session creation, and inference succeeds.

For comparison, the NVIDIA TensorRT RTX Plugin EP exposes a `DEFAULT` memory info and `GetSharedAllocator()` returns a usable allocator immediately after plugin registration, before any explicit `CreateSharedAllocator()` call.

Related background: #25785. That issue describes the earlier state where WebGPU did not expose device memory info. In this report, the current WebGPU Plugin EP does expose valid device memory info, but the documented default shared allocator is still missing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the RegisterExecutionProviderLibrary, GetMemoryInfo, GetSharedAllocator, and CreateSharedAllocator entry points, using the supplied minimal reproduction to trace WebGPU allocator registration. Compare the behavior with the TensorRT RTX Plugin EP. Done means GetSharedAllocator(memory_info) returns a usable allocator immediately after registration, without explicit CreateSharedAllocator().

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.