JuliaGPU / JuliaGPU/KernelAbstractions.jl

Inconsistent results across platforms and datatypes on CPU backend

Open
#468 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
523
Forks
88
Avg merge
1d 11h
Merged PRs (30d)
25

Description

I am trying to write a simple reduction kernel, which seems to only work for certain datatypes.
The datatypes it works for is inconsistent across platforms.

Here is the kernel, which computes the sum of squares for a vector of 6000 ones -

using KernelAbstractions
using Atomix: @atomic

@kernel function self_dot_kernel!(result, @Const(vec))
  BLOCK_SIZE = @uniform @groupsize()[1]
  I = @index(Global, Linear)
  li = @index(Local, Linear)

  intermediateresults = @localmem eltype(vec) (BLOCK_SIZE)

  if I <= length(vec)
    intermediateresults[li] = vec[I] * vec[I]
  else
    intermediateresults[li] = 0
  end
  @synchronize()

  for i in [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
    if li + i <= BLOCK_SIZE
      intermediateresults[li] += intermediateresults[li+i]
    end
    @synchronize()
  end


  @synchronize()
  if li == 1
    @atomic result[1] += intermediateresults[1]
  end
end

backend = CPU()
types = [Float16, Float32, Float64, Int16, Int32, Int64, UInt16, UInt32, UInt64]
kernel! = self_dot_kernel!(backend, 32)
platform = Sys.MACHINE

for T in types
  X = KernelAbstractions.ones(backend, T, 6000)
  println("\n=== Type: $T ($platform) ==\n")

  for i in 1:6
    dotresult = KernelAbstractions.zeros(backend, T, 1)
    kernel!(dotresult, X, ndrange=length(X))
    KernelAbstractions.synchronize(backend)

    if i % 3 == 0
      println("  $(dotresult[1])")
    else
      print("  $(dotresult[1]),")
    end
  end
end

And here are some outputs, ran using julia 1.10.2 -

=== Type: Float16 (arm64-apple-darwin23.2.0) ==

  6.0e3,  6.0e3,  5.996e3
  6.0e3,  7.424e3,  6.0e3

=== Type: Float32 (arm64-apple-darwin23.2.0) ==

  6000.0,  6000.0,  6000.0
  6000.0,  6000.0,  6000.0

=== Type: Float64 (arm64-apple-darwin23.2.0) ==

  6000.0,  6000.0,  6000.0
  6000.0,  6000.0,  6000.0

=== Type: Int16 (arm64-apple-darwin23.2.0) ==

  6001,  6004,  6003
  6000,  6003,  6001

=== Type: Int32 (arm64-apple-darwin23.2.0) ==

  476212432,  49993592,  6001
  6004,  6004,  49993592

=== Type: Int64 (arm64-apple-darwin23.2.0) ==

  6000,  6004,  6000
  4547881696,  4546395904,  6000

=== Type: UInt16 (arm64-apple-darwin23.2.0) ==

  6003,  6003,  6003
  6003,  6003,  6003

=== Type: UInt32 (arm64-apple-darwin23.2.0) ==

  254459536,  230300864,  468297712
  6000,  6000,  50026560

=== Type: UInt64 (arm64-apple-darwin23.2.0) ==

  6038,  6001,  4551972480
  4554654336,  6001,  6000

Float32, and Float64 always work on apple-aarch64, making me think that there's nothing wrong with the algorithm. And on windows-x64 -

=== Type: Float16 (x86_64-w64-mingw32) ==

  6.0e3,  6.0e3,  6.0e3
  6.0e3,  6.0e3,  6.0e3

=== Type: Float32 (x86_64-w64-mingw32) ==

  6000.0,  6000.0,  6000.0
  -2.7825533e35,  6000.0,  6000.0

=== Type: Float64 (x86_64-w64-mingw32) ==

  6000.0,  6000.0,  6000.0
  6000.0,  6000.0,  6000.0

=== Type: Int16 (x86_64-w64-mingw32) ==

  6004,  6000,  6001
  6002,  6000,  6003

=== Type: Int32 (x86_64-w64-mingw32) ==

  -556641600,  -150726792,  -150726792
  -108881600,  -91191952,  6041

=== Type: Int64 (x86_64-w64-mingw32) ==

  42949678970,  1846738422272,  1846692115360
  1846740674704,  6000,  1846740275056

=== Type: UInt16 (x86_64-w64-mingw32) ==

  6002,  6027,  6003
  6004,  6003,  6004

=== Type: UInt32 (x86_64-w64-mingw32) ==

  6011,  4144240504,  4165309344
  4170999664,  569204,  6000

=== Type: UInt64 (x86_64-w64-mingw32) ==

  140724418903328,  1846746033440,  1846275974656
  6000,  6009,  1846685210488

On windows-x64, only Float64 consistently works. What is going on?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied Julia reproducer with the CPU() backend and compare the @atomic reduction across the listed datatypes and platforms. Inspect the CPU backend's handling of @localmem, synchronization, and atomic updates. Done means the reduction consistently returns 6000 for every supported datatype and platform, or clearly documents unsupported cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.