JuliaGPU / JuliaGPU/GPUCompiler.jl
Cthulhu uses InteractiveUtils.code_llvm instead of GPUCompiler.jl
- Dominant language
- Julia
- Stars
- 187
- Forks
- 68
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 28
Description
As discussed on Slack (#gpu), the LLVM code output shown after pressing `L` in an Cthulhu interactive session is run through the wrong pipeline, resulting in host-side optimisations/… being applied to it.
---
### Real-world example
For instance, consider this
```julia
using Metal
@metal (() -> (rand(Float32); nothing))()
```
which fails, as scalar rand() is not implemented on Metal yet. The error message is
```
ERROR: InvalidIRError: compiling MethodInstance for (::var"#3#4")() resulted in invalid LLVM IR
Reason: unsupported call to an unknown function (call to julia.get_pgcstack)
Hint: catch this exception as `err` and call `code_typed(err; interactive = true)` to introspect the erronous code with Cthulhu.jl
[…]
```
and running the suggested code
```julia
julia> try
@metal (() -> (rand(Float32); nothing))()
catch err
code_typed(err; interactive=true)
end
```
initally gives the correct `code_typed()` output
```
(::var"#3#4")() @ Main REPL[10]:2
∘ ── %0 = invoke #13()::Core.Const(nothing)
2 1 ── %1 = $(Expr(:foreigncall, :(:jl_get_current_task), Ref{Task}, svec(), 0, :(:ccall)))::Task │╻╷╷╷╷╷╷╷ rand
2 ── nothing::Nothing
[…]
```
(here, the RNG tries to access TLS state, hence the reference to `jl_get_current_task`, which gets lowered to the offending `julia.get_pgcstack`).
However, pressing `L` to look at the actual LLVM IR gives the following (debug info turned off):
```
define swiftcc void @"julia_#1_2572"({}*** nonnull swiftself %0) #0 {
top:
%ptls_field4 = getelementptr inbounds {}**, {}*** %0, i64 2
%1 = bitcast {}*** %ptls_field4 to i64***
%ptls_load56 = load i64**, i64*** %1, align 8
%2 = getelementptr inbounds i64*, i64** %ptls_load56, i64 2
%safepoint = load i64*, i64** %2, align 8
fence syncscope("singlethread") seq_cst
%3 = load volatile i64, i64* %safepoint, align 8
fence syncscope("singlethread") seq_cst
%4 = getelementptr inbounds {}**, {}*** %0, i64 -7
%5 = bitcast {}*** %4 to i64*
%6 = load i64, i64* %5, align 8
%7 = getelementptr inbounds {}**, {}*** %0, i64 -6
%8 = bitcast {}*** %7 to i64*
%9 = load i64, i64* %8, align 8
%10 = getelementptr inbounds {}**, {}*** %0, i64 -5
%11 = bitcast {}*** %10 to i64*
%12 = load i64, i64* %11, align 8
%13 = getelementptr inbounds {}**, {}*** %0, i64 -4
%14 = bitcast {}*** %13 to i64*
%15 = load i64, i64* %14, align 8
%16 = shl i64 %9, 17
%17 = xor i64 %12, %6
%18 = xor i64 %15, %9
%19 = xor i64 %17, %9
%20 = xor i64 %18, %6
%21 = xor i64 %17, %16
%22 = call i64 @llvm.fshl.i64(i64 %18, i64 %18, i64 45)
store i64 %20, i64* %5, align 8
store i64 %19, i64* %8, align 8
store i64 %21, i64* %11, align 8
store i64 %22, i64* %14, align 8
ret void
}
```
which seems to have been run through the host-side pipeline, as the `julia.get_pgcstack` call has been lowered into an extra argument, etc. This is obviously not desirable in any case, but here it is particularly confusing, as the host-side pipeline has removed the offending code.
---
@vchuravy has identified the root cause for this in the `code_llvm` for the host pipeline being called here: https://github.com/JuliaDebug/Cthulhu.jl/blob/3cd0baf586ab07f18ecf9c6f040e6764e8155322/src/codeview.jl#L26
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at Cthulhu.jl's src/codeview.jl line 26, where the issue identifies the host-pipeline code_llvm call. Compare that path with the GPUCompiler pipeline, then reproduce the Metal rand example and inspect the LLVM output shown after pressing L. Done means the interactive LLVM view uses the appropriate GPU pipeline without applying host-side transformations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100