[GSD-13095] SYCL global_mem_size reports PF total LMEM instead of VF partition on some SR-IOV VFs (Arc Pro B70)
- Dominant language
- C++
- Stars
- 1.4k
- Forks
- 300
- PR merge metrics
- No merged PRs in 30d
Description
### Pre-submission Checklist
- [x] I am using the latest GPU driver version ([releases](https://github.com/intel/compute-runtime/releases))
- [x] I have searched for similar issues and found none
### GPU Hardware
Intel Arc Pro B70 (BMG-G31, PCI ID 8086:e223)
### DRI Devices Information
2x Arc Pro B70 PFs with 4 SR-IOV VFs each (8 VFs total):
```
$ ls -la /dev/dri/by-path/
pci-0000:18:00.0-card -> ../card1 # GPU1 PF
pci-0000:18:00.0-render -> ../renderD128
pci-0000:18:00.1-card -> ../card3 # GPU1 VF1
pci-0000:18:00.1-render -> ../renderD130
pci-0000:18:00.2-card -> ../card4 # GPU1 VF2
pci-0000:18:00.2-render -> ../renderD131
pci-0000:18:00.3-card -> ../card5 # GPU1 VF3
pci-0000:18:00.3-render -> ../renderD132
pci-0000:18:00.4-card -> ../card6 # GPU1 VF4
pci-0000:18:00.4-render -> ../renderD133
pci-0000:54:00.0-card -> ../card2 # GPU2 PF
pci-0000:54:00.0-render -> ../renderD129
pci-0000:54:00.1-card -> ../card7 # GPU2 VF1
pci-0000:54:00.1-render -> ../renderD134
pci-0000:54:00.2-card -> ../card8 # GPU2 VF2
pci-0000:54:00.2-render -> ../renderD135
pci-0000:54:00.3-card -> ../card9 # GPU2 VF3
pci-0000:54:00.3-render -> ../renderD136
pci-0000:54:00.4-card -> ../card10 # GPU2 VF4
pci-0000:54:00.4-render -> ../renderD137
```
### GPU Detailed Information (lspci output)
```
18:00.0 VGA compatible controller: Intel Corporation Battlemage G31 [Intel Graphics] (prog-if 00 [VGA controller])
Subsystem: Intel Corporation Device 1701
Region 0: Memory at 207804000000 (64-bit, prefetchable) [size=16M]
Region 2: Memory at 206800000000 (64-bit, prefetchable) [size=32G]
Capabilities: [420 v1] Physical Resizable BAR
BAR 2: current size: 32GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB 16GB 32GB
Capabilities: [220 v1] Virtual Resizable BAR
BAR 2: current size: 8GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB 16GB 32GB
Capabilities: [320 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable+ Migration- Interrupt- MSE+ ARIHierarchy+
Initial VFs: 4, Total VFs: 4, Number of VFs: 4, Function Dependency Link: 00
VF offset: 1, stride: 1, Device ID: e223
Kernel driver in use: xe
```
All VF BAR2 sizes are identical at 8 GiB (`stat -c %s /sys/bus/pci/devices/0000:XX:00.Y/resource2` = 8589934592 for all VFs).
### Driver Version
26.18.38308.4
### Installed GPU Driver Packages
Tested inside `docker.io/intel/oneapi:2026.0.0-devel-rockylinux10` container with packages upgraded to latest:
```
intel-compute-runtime-26.18.38308.4-1.el10_2.x86_64
intel-level-zero-26.18.38308.4-1.el10_2.x86_64
intel-igc-2.34.4-4.el10_2.x86_64
intel-igc-libs-2.34.4-4.el10_2.x86_64
intel-gmmlib-22.8.1-2.el10_1.x86_64
intel-opencl-26.18.38308.4-1.el10_2.x86_64
intel-ocloc-26.18.38308.4-1.el10_2.x86_64
```
### Driver Installation Details
Packages installed from the Rocky Linux 10 Intel GPU repos (`dnf upgrade intel-level-zero intel-compute-runtime intel-igc intel-igc-libs intel-opencl intel-ocloc oneapi-level-zero`). Workloads run as Kubernetes pods with DRA-assigned `/dev/dri/renderD*` devices (one VF per pod).
### Linux Distribution
Fedora / RHEL
### Other Linux Distribution
RHEL CoreOS 9.8 (OpenShift 4.22.2). Userspace runs inside Rocky Linux 10 containers.
### Kernel Version & Boot Parameters
```
$ uname -r
5.14.0-687.15.1.el9_8.x86_64
$ cat /proc/cmdline
... pci=realloc intel_iommu=on iommu=pt ...
$ lsmod | grep xe
xe 3985408 4
```
### Actual Behavior
`sycl::info::device::global_mem_size` reports the full PF LMEM size (~31023 MiB) on some SR-IOV VFs instead of the correct per-VF partition (~7719 MiB). Affected VFs vary per GPU — on this system, GPU1 VF1 and VF4 are affected while GPU1 VF2-3 and all GPU2 VFs report correctly.
Tested by deploying one SYCL pod per VF (PCI-address-pinned via DRA ResourceClaims), each running:
```cpp
auto dev = q.get_device();
std::cout << "Memory: " << dev.get_info() / (1024*1024) << " MiB" << std::endl;
```
Results:
| VF | PCI Address | SYCL global_mem_size | Expected |
|---|---|---|---|
| GPU1 VF1 | `0000:18:00.1` | **31023 MiB** | 7719 MiB |
| GPU1 VF2 | `0000:18:00.2` | 7719 MiB | 7719 MiB |
| GPU1 VF3 | `0000:18:00.3` | 7719 MiB | 7719 MiB |
| GPU1 VF4 | `0000:18:00.4` | **31023 MiB** | 7719 MiB |
| GPU2 VF1 | `0000:54:00.1` | 7719 MiB | 7719 MiB |
| GPU2 VF2 | `0000:54:00.2` | 7719 MiB | 7719 MiB |
| GPU2 VF3 | `0000:54:00.3` | 7719 MiB | 7719 MiB |
| GPU2 VF4 | `0000:54:00.4` | 7719 MiB | 7719 MiB |
The values 7719 and 31023 are consistent with a 0.95 headroom factor applied to the correct VF size (8126 MiB) and the PF visible size (32656 MiB), respectively: `8126 * 0.95 ≈ 7719`, `32656 * 0.95 ≈ 31023`.
**The kernel is not the source.** The xe driver's TTM resource manager reports the correct size for all VFs, confirmed via debugfs:
```
# cat /sys/kernel/debug/dri/{card}/vram0_mm | grep 'man size'
card1 (0000:18:00.0) [PF]: man size:34242297856 (32656 MiB)
card3 (0000:18:00.1) [VF]: man size:8520728576 (8126 MiB) <-- correct
card4 (0000:18:00.2) [VF]: man size:8520728576 (8126 MiB)
card5 (0000:18:00.3) [VF]: man size:8520728576 (8126 MiB)
card6 (0000:18:00.4) [VF]: man size:8520728576 (8126 MiB) <-- correct
card7 (0000:54:00.1) [VF]: man size:8520728576 (8126 MiB)
...all VFs identical...
```
The `DRM_IOCTL_XE_DEVICE_QUERY` (MEM_REGIONS) response from the kernel is correct for all VFs. The misreporting happens in the compute-runtime/L0 userspace layer.
dmesg also confirms correct VF LMEM provisioning by the PF:
```
xe 0000:18:00.0: [drm] Tile0: GT0: PF: VF1..VF4 provisioned with 8520728576 (7.94 GiB) LMEM
xe 0000:54:00.0: [drm] Tile0: GT0: PF: VF1..VF4 provisioned with 8520728576 (7.94 GiB) LMEM
```
### Expected Behavior
`sycl::info::device::global_mem_size` should report the VF's actual LMEM partition (~7719 MiB after headroom) on all VFs, matching the kernel-reported `man->size` of 8520728576 bytes.
### Reproduction Rate
Always reproduces - 100%
### Steps to Reproduce
1. System with 2x Intel Arc Pro B70 GPUs, xe driver loaded
2. Create 4 SR-IOV VFs per GPU: `echo 4 > /sys/bus/pci/devices/0000:XX:00.0/sriov_numvfs`
3. Run a SYCL program that queries `sycl::info::device::global_mem_size` on each VF independently (one process per VF, each seeing only its own `/dev/dri/renderD*`)
4. Compare reported memory across VFs — some report ~31023 MiB (PF value) instead of ~7719 MiB (VF value)
### oneAPI Version (if applicable)
```
$ icpx --version
Intel(R) oneAPI DPC++/C++ Compiler 2026.0.0 (2026.0.0.20260331)
Target: x86_64-unknown-linux-gnu
```
### Additional Notes
- Compute performance is **not affected** — all VFs deliver identical GFLOPS regardless of the reported memory value. The actual LMEM allocation and compute work correctly; only the reporting is wrong.
- vLLM inference also works correctly on affected VFs — `Free memory on device (7.03/7.54 GiB)` is reported accurately by the vLLM memory profiler path, which uses `torch.xpu.mem_get_info()` rather than `global_mem_size`.
- The pattern of which VFs are affected is not consistent with a "first VF" or "last VF" rule — it's VF1 + VF4 on GPU1 only, while GPU2 is unaffected.
Contributor guide
Research direction
Start at the SYCL device-info query for global_mem_size in the compute-runtime/Level Zero userspace path, then compare its input with the DRM_IOCTL_XE_DEVICE_QUERY MEM_REGIONS response and the xe debugfs vram0_mm values described here. Reproduce across the listed SR-IOV VFs and verify that every VF reports about 7719 MiB rather than the PF value of about 31023 MiB.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- backend-api-design
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100