KhronosGroup / KhronosGroup/OpenCL-Docs
SPIR-V environment spec clarification for OpGroup execution scope
- Dominant language
- Python
- Stars
- 420
- Forks
- 131
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 11
Description
Hi,
I've encountered some potentially confusing behaviour of the sub-group and work-group instructions.
The various spir-v `OpGroup.*` operations take an execution argument that is a scope and the OpenCL environment spec places some restrictions on the values that the scope can be: https://www.khronos.org/registry/OpenCL/specs/3.0-unified/html/OpenCL_Env.html#validation-rules
```
For the Group and Subgroup Instructions, Scope for Execution must be one of:
Workgroup
For OpenCL 2.0, OpenCL 2.1, OpenCL 2.2, or OpenCL 3.0 devices supporting Work Group Collective Functions (where CL_DEVICE_WORK_GROUP_COLLECTIVE_FUNCTIONS_SUPPORT is CL_TRUE).
Subgroup
For OpenCL 2.1, OpenCL 2.2, or OpenCL 3.0 devices supporting Subgroups (where CL_DEVICE_MAX_NUM_SUB_GROUPS is not 0)
```
The spir-v spec for the group operations also has some wording about what values the scope argument can be https://www.khronos.org/registry/SPIR-V/specs/unified1/SPIRV.html#Group:
```
Execution is a Scope. It must be either Workgroup or Subgroup.
```
As far as I can tell (and I may have just missed the relevant wording here, apologies if that's the case) the OpenCL environment spec does not seem to do is require that the scope argument be an `OpConstant`. This seems slightly odd for OpenCL, since the builtins these would normally map to encode their scope in the symbol name, making it inherently constant, rather than passing it as a dynamic argument e.g. `OpGroupIAdd` for the `Reduce` operation could either get mapped to `sub_group_reduce_add` or `work_group_reduce_add` (in theory this could be resolved using some switch based on the value of the scope, but it's not as nice as a direct mapping).
Out of curiosity, I wrote some spir-v that used a dynamic scope as an operand to `OpGroupIAdd`, fed it into the spirv-llvm bidirectional translator (https://github.com/KhronosGroup/SPIRV-LLVM-Translator) and got assert. It looks like the spir-v-llvm translator makes the hard assumption that the scope argument will be constant. I'm not particularly familiar with the project, but from a quick debugging session it seems when handling the group operations it produces a call to a spirv builtin and tries to extract the scope from the first argument here: https://github.com/KhronosGroup/SPIRV-LLVM-Translator/blob/master/lib/SPIRV/SPIRVToOCL.cpp#L1164, which it eventually cast to a `ConstantInt` here: https://github.com/KhronosGroup/SPIRV-LLVM-Translator/blob/master/lib/SPIRV/SPIRVUtil.cpp#L386. I'm happy to share the example here if that helps.
So I think if all the above is correct, there are two possibilities here:
1. There is a spec bug and the OpenCL SPIR-V environment spec should require scopes to group operations be compile time constants.
2. There is a bug in the bidirectional translator, and the scope operands should not be assumed constant when generating ir from spirv-ll in the OpenCL context.
For case 1. I'm happy to open a PR for the environment spec, for 2. I can close this issue and open one with the SPIRV-LLVM-Translator project.
Cheers,
Jack.
Contributor guide
Research direction
Read the OpenCL environment specification validation rules and the SPIR-V Group instruction requirements linked in the issue. Compare them with the translator assumptions in lib/SPIRV/SPIRVToOCL.cpp and lib/SPIRV/SPIRVUtil.cpp. Done means resolving whether scopes must be OpConstants and documenting or routing the required change to the appropriate project.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100