mudler / mudler/vllm.cpp

[Vulkan][32-bit] non-dispatchable handles do not fit in void* on i586

Open
#3,182 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Problem

The Vulkan backend does not compile on 32-bit x86/i586. In src/vt/vulkan/vulkan_context.cpp, it asserts that Vulkan handles fit in void*:

static_assert(sizeof(VkBuffer) <= sizeof(void*), "VkBuffer must fit in void*");
static_assert(sizeof(VkDeviceMemory) <= sizeof(void*), "VkDeviceMemory must fit in void*");

With the Vulkan headers on i586, the assertions fail because the comparison is 8 <= 4. Vulkan non-dispatchable handles such as VkBuffer and VkDeviceMemory are 64-bit values even when the host pointer size is 32-bit.

The backend then packs these handles through void* in Pack/Unpack, and the public backend context stores buffer and memory handles as void*, so simply removing the assertions would not be sufficient.

Reproducer

Build v0.0.2 on i586 with:

-DVLLM_CPP_VULKAN=ON

The build fails in src/vt/vulkan/vulkan_context.cpp with:

error: static assertion failed: VkBuffer must fit in void*
the comparison reduces to (8 <= 4)
error: static assertion failed: VkDeviceMemory must fit in void*
the comparison reduces to (8 <= 4)

Proposed direction

Use an architecture-independent opaque handle representation, such as a dedicated 64-bit storage type for non-dispatchable handles, and avoid packing them into void*. Dispatchable pointer handles can remain pointer-sized.

The openSUSE package currently disables the Vulkan backend on 32-bit x86 as a downstream workaround.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/vt/vulkan/vulkan_context.cpp by tracing the VkBuffer and VkDeviceMemory static assertions, then inspect the Pack/Unpack paths and the public backend context fields that store these handles. Build v0.0.2 with -DVLLM_CPP_VULKAN=ON on i586; done means the Vulkan backend compiles without routing non-dispatchable handles through void*.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.