Allocating device local + host visible memory more than GPU maximum
- Dominant language
- C++
- Stars
- 3.9k
- Forks
- 1k
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 47
Description
### What happened?
The dispatch with index 1122 is giving a problem, i.e. the output sometimes has -INF or NaNs. On a close look, the dispatch seems to be BatchMatmul. I tried recreating the issue by passing the input from the previous dispatch and running BatchMatmul in isolation (attached the python file for the repro), but there's no overflow.
Turning on Vulkan validation layers while running the whole IR throws these errors:
``` [VULKAN] ! Validation Error: [ VUID-vkAllocateMemory-pAllocateInfo-01713 ] Object 0: handle = 0x55ba3e384b30, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xe9a2b96f | vkAllocateMemory: attempting to allocate 1698693120 bytes from heap 2,but size of that heap is only 257949696 bytes. The Vulkan spec states: pAllocateInfo->allocationSize must be less than or equal to VkPhysicalDeviceMemoryProperties::memoryHeaps[memindex].size where memindex = VkPhysicalDeviceMemoryProperties::memoryTypes[pAllocateInfo->memoryTypeIndex].heapIndex as returned by vkGetPhysicalDeviceMemoryProperties for the VkPhysicalDevice that device was created from (https://vulkan.lunarg.com/doc/view/1.3.239.0/linux/1.3-extensions/vkspec.html#VUID-vkAllocateMemory-pAllocateInfo-01713) ```
``` [VULKAN] ! Validation Error: [ VUID-VkShaderModuleCreateInfo-pCode-04147 ] Object 0: handle = 0x555c08b53230, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x3d492883 | vkCreateShaderModule(): The SPIR-V Extension (SPV_NV_cooperative_matrix) was declared, but none of the requirements were met to use it. The Vulkan spec states: If pCode declares any of the SPIR-V extensions listed in the SPIR-V Environment appendix, one of the corresponding requirements must be satisfied (https://vulkan.lunarg.com/doc/view/1.3.239.0/linux/1.3-extensions/vkspec.html#VUID-VkShaderModuleCreateInfo-pCode-04147) ```
### Steps to reproduce your issue
Model IR: https://storage.googleapis.com/shark-public/prashant/unet_check/unet_check.mlir
Dot graph: https://storage.googleapis.com/shark-public/prashant/unet_check/dispatch.dot
Python script and inputs: https://storage.googleapis.com/shark-public/prashant/unet_check/overflow_ir.py, https://storage.googleapis.com/shark-public/prashant/unet_check/out.npy
Compile command :-
`iree-compile --iree-input-type=none --iree-hal-target-backends=vulkan -iree-vulkan-target-triple=ampere-rtx3090-linux --iree-stream-resource-index-bits=64 --iree-vm-target-index-bits=64 --iree-preprocessing-pass-pipeline='builtin.module(func.func(iree-flow-detach-elementwise-from-named-ops,iree-flow-convert-1x1-filter-conv2d-to-matmul,iree-preprocessing-convert-conv2d-to-img2col,iree-preprocessing-pad-linalg-ops{pad-size=32}))' unet_check.mlir -o out.vmfb`
Run command :-
`iree-run-module --device=vulkan --function=forward --input=2x4x96x96xf16=0.5 --input=1xf16=1.0 --input=2x77x1024xf16=0.5 --module=out.vmfb --vulkan_debug_utils=true --vulkan_debug_verbosity=4 --vulkan_validation_layers=true `
### What component(s) does this issue relate to?
Runtime
### Version information
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.