oneapi-src / oneapi-src/unified-runtime

[CUDA] Misleading error handling for hasExceededMaxRegistersPerBlock

Open
#1,308 0 comments 0 reactions 1 assignee View on GitHub

@GeorgeWeb is already working on this.

Since Feb 6, 2024.

cuda
Dominant language
C++
Stars
57
Forks
120
Avg merge
1d 14h
Merged PRs (30d)
1

Description

Coming from #1299 which originally included a change of the error code, but upon further discussion with @GeorgeWeb we agreed the error handling improvement should be a separate PR, paired with correct handling of the changed error code in intel/llvm.

The issue is that when a user submits a kernel that has a reasonable work group size, but exceeds the available number of registers on the device, the setKernelParams function in source/adapters/cuda/enqueue.cpp returns UR_RESULT_ERROR_INVALID_WORK_GROUP_SIZE which faces user with the following error:

terminate called after throwing an instance of 'sycl::_V1::nd_range_error'
  what():  Non-uniform work-groups are not supported by the target device -54 (PI_ERROR_INVALID_WORK_GROUP_SIZE)

even if they submit perfectly uniform work groups. This came up initially in https://github.com/intel/llvm/issues/12363 where the global and local sizes were 2048x788, 1024x1. The error is simply wrong in this case - an "out of resources" error should be reported instead.

Side note / another related but separate issue:
It also seems the intel/llvm message for UR_RESULT_ERROR_INVALID_WORK_GROUP_SIZE is misleading in any case, because "invalid work group size" does not always mean non-uniform work-groups, it could be e.g. exceeding the max work group size.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.