iree-org / iree-org/iree

Allocating device local + host visible memory more than GPU maximum

Open
#12,876 15 comments 0 reactions 1 assignee Assigned to @pashu123 View on GitHub
bug 🐞 hal/vulkan
Dominant language
C++
Stars
3.9k
Forks
1k
Avg merge
4d 16h
Merged PRs (30d)
47

Description

### What happened?

The dispatch with index 1122 is giving a problem, i.e. the output sometimes has -INF or NaNs. On a close look, the dispatch seems to be BatchMatmul. I tried recreating the issue by passing the input from the previous dispatch and running BatchMatmul in isolation (attached the python file for the repro), but there's no overflow.

Turning on Vulkan validation layers while running the whole IR throws these errors:

``` [VULKAN] ! Validation Error: [ VUID-vkAllocateMemory-pAllocateInfo-01713 ] Object 0: handle = 0x55ba3e384b30, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xe9a2b96f | vkAllocateMemory: attempting to allocate 1698693120 bytes from heap 2,but size of that heap is only 257949696 bytes. The Vulkan spec states: pAllocateInfo->allocationSize must be less than or equal to VkPhysicalDeviceMemoryProperties::memoryHeaps[memindex].size where memindex = VkPhysicalDeviceMemoryProperties::memoryTypes[pAllocateInfo->memoryTypeIndex].heapIndex as returned by vkGetPhysicalDeviceMemoryProperties for the VkPhysicalDevice that device was created from (https://vulkan.lunarg.com/doc/view/1.3.239.0/linux/1.3-extensions/vkspec.html#VUID-vkAllocateMemory-pAllocateInfo-01713) ```

``` [VULKAN] ! Validation Error: [ VUID-VkShaderModuleCreateInfo-pCode-04147 ] Object 0: handle = 0x555c08b53230, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x3d492883 | vkCreateShaderModule(): The SPIR-V Extension (SPV_NV_cooperative_matrix) was declared, but none of the requirements were met to use it. The Vulkan spec states: If pCode declares any of the SPIR-V extensions listed in the SPIR-V Environment appendix, one of the corresponding requirements must be satisfied (https://vulkan.lunarg.com/doc/view/1.3.239.0/linux/1.3-extensions/vkspec.html#VUID-VkShaderModuleCreateInfo-pCode-04147) ```

### Steps to reproduce your issue

Model IR: https://storage.googleapis.com/shark-public/prashant/unet_check/unet_check.mlir
Dot graph: https://storage.googleapis.com/shark-public/prashant/unet_check/dispatch.dot

Python script and inputs: https://storage.googleapis.com/shark-public/prashant/unet_check/overflow_ir.py, https://storage.googleapis.com/shark-public/prashant/unet_check/out.npy

Compile command :-
`iree-compile --iree-input-type=none --iree-hal-target-backends=vulkan -iree-vulkan-target-triple=ampere-rtx3090-linux --iree-stream-resource-index-bits=64 --iree-vm-target-index-bits=64 --iree-preprocessing-pass-pipeline='builtin.module(func.func(iree-flow-detach-elementwise-from-named-ops,iree-flow-convert-1x1-filter-conv2d-to-matmul,iree-preprocessing-convert-conv2d-to-img2col,iree-preprocessing-pad-linalg-ops{pad-size=32}))' unet_check.mlir -o out.vmfb`

Run command :-
`iree-run-module --device=vulkan --function=forward --input=2x4x96x96xf16=0.5 --input=1xf16=1.0 --input=2x77x1024xf16=0.5 --module=out.vmfb --vulkan_debug_utils=true --vulkan_debug_verbosity=4 --vulkan_validation_layers=true `

### What component(s) does this issue relate to?

Runtime

### Version information

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.