JuliaGPU / JuliaGPU/KernelAbstractions.jl
WaterLily performance regression on CPU multi-threading backend for Julia 1.12.0
Open
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 523
- Forks
- 88
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 25
Description
Here are some WaterLily benchmarks with Float32 comparing Julia 1.11.5 with 1.12.0. The KA CPU multi-threading backend shows performance regression on Julia 1.12.0, specially on CPUx04 where allocations are increasing significantly.
To run your own benchmarks, please use WaterLily-Benchmarks and
sh benchmark.sh -v "1.11.5 release" -w "master" -t "1 2 4" -b "Array CuArray" -c "tgv jelly" -p "6,7 5,6" -s "100 100" -ft "Float32 Float32"
julia --project compare.jl
(Note that env var $WATERLILY_DIR needs to be set to your local WaterLily directory, or alternatively pass -wd "my/waterlily/dir" into benchmarh.sh).
--
Benchmark environment: tgv sim_step! (max_steps=100)
▶ log2p = 6
┌────────────┬───────────┬────────┬───────────┬─────────────┬────────┬──────────┬──────────────────┬──────────┐
│ Backend │ WaterLily │ Julia │ Precision │ Allocations │ GC [%] │ Time [s] │ Cost [ns/DOF/dt] │ Speed-up │
├────────────┼───────────┼────────┼───────────┼─────────────┼────────┼──────────┼──────────────────┼──────────┤
│ CPUx01 │ master │ 1.11.5 │ Float32 │ 1807 │ 0.00 │ 4.16 │ 158.85 │ 1.00 │
│ CPUx01 │ master │ 1.12.0 │ Float32 │ 1807 │ 0.00 │ 4.28 │ 163.16 │ 0.97 │
│ CPUx02 │ master │ 1.11.5 │ Float32 │ 1687271 │ 0.00 │ 4.82 │ 183.72 │ 0.86 │
│ CPUx02 │ master │ 1.12.0 │ Float32 │ 1651033 │ 0.00 │ 5.55 │ 211.89 │ 0.75 │
│ CPUx04 │ master │ 1.11.5 │ Float32 │ 2334995 │ 0.00 │ 2.86 │ 109.25 │ 1.45 │
│ CPUx04 │ master │ 1.12.0 │ Float32 │ 193455597 │ 9.58 │ 4.47 │ 170.65 │ 0.93 │
│ GPU-NVIDIA │ master │ 1.11.5 │ Float32 │ 3041056 │ 0.00 │ 0.60 │ 22.73 │ 6.99 │
│ GPU-NVIDIA │ master │ 1.12.0 │ Float32 │ 2208405 │ 0.00 │ 0.59 │ 22.35 │ 7.11 │
└────────────┴───────────┴────────┴───────────┴─────────────┴────────┴──────────┴──────────────────┴──────────┘
▶ log2p = 7
┌────────────┬───────────┬────────┬───────────┬─────────────┬────────┬──────────┬──────────────────┬──────────┐
│ Backend │ WaterLily │ Julia │ Precision │ Allocations │ GC [%] │ Time [s] │ Cost [ns/DOF/dt] │ Speed-up │
├────────────┼───────────┼────────┼───────────┼─────────────┼────────┼──────────┼──────────────────┼──────────┤
│ CPUx01 │ master │ 1.11.5 │ Float32 │ 1807 │ 0.00 │ 25.66 │ 122.35 │ 1.00 │
│ CPUx01 │ master │ 1.12.0 │ Float32 │ 1807 │ 0.00 │ 26.23 │ 125.08 │ 0.98 │
│ CPUx02 │ master │ 1.11.5 │ Float32 │ 1571048 │ 0.00 │ 29.16 │ 139.06 │ 0.88 │
│ CPUx02 │ master │ 1.12.0 │ Float32 │ 1537319 │ 0.00 │ 31.40 │ 149.75 │ 0.82 │
│ CPUx04 │ master │ 1.11.5 │ Float32 │ 2175214 │ 0.00 │ 17.07 │ 81.41 │ 1.50 │
│ CPUx04 │ master │ 1.12.0 │ Float32 │ 951576349 │ 9.22 │ 24.21 │ 115.46 │ 1.06 │
│ GPU-NVIDIA │ master │ 1.11.5 │ Float32 │ 2784710 │ 0.00 │ 3.03 │ 14.44 │ 8.47 │
│ GPU-NVIDIA │ master │ 1.12.0 │ Float32 │ 2329316 │ 0.00 │ 3.12 │ 14.87 │ 8.23 │
└────────────┴───────────┴────────┴───────────┴─────────────┴────────┴──────────┴──────────────────┴──────────┘
Benchmark environment: jelly sim_step! (max_steps=100)
▶ log2p = 5
┌────────────┬───────────┬────────┬───────────┬─────────────┬────────┬──────────┬──────────────────┬──────────┐
│ Backend │ WaterLily │ Julia │ Precision │ Allocations │ GC [%] │ Time [s] │ Cost [ns/DOF/dt] │ Speed-up │
├────────────┼───────────┼────────┼───────────┼─────────────┼────────┼──────────┼──────────────────┼──────────┤
│ CPUx01 │ master │ 1.11.5 │ Float32 │ 7107 │ 0.00 │ 3.49 │ 266.22 │ 1.00 │
│ CPUx01 │ master │ 1.12.0 │ Float32 │ 6307 │ 0.00 │ 3.54 │ 269.76 │ 0.99 │
│ CPUx02 │ master │ 1.11.5 │ Float32 │ 3244853 │ 0.00 │ 5.30 │ 404.21 │ 0.66 │
│ CPUx02 │ master │ 1.12.0 │ Float32 │ 3174214 │ 0.00 │ 6.16 │ 470.05 │ 0.57 │
│ CPUx04 │ master │ 1.11.5 │ Float32 │ 4491947 │ 0.67 │ 3.37 │ 257.31 │ 1.03 │
│ CPUx04 │ master │ 1.12.0 │ Float32 │ 299427100 │ 10.59 │ 5.44 │ 414.85 │ 0.64 │
│ GPU-NVIDIA │ master │ 1.11.5 │ Float32 │ 5805841 │ 1.49 │ 0.98 │ 74.80 │ 3.56 │
│ GPU-NVIDIA │ master │ 1.12.0 │ Float32 │ 4179289 │ 0.00 │ 0.89 │ 68.16 │ 3.91 │
└────────────┴───────────┴────────┴───────────┴─────────────┴────────┴──────────┴──────────────────┴──────────┘
▶ log2p = 6
┌────────────┬───────────┬────────┬───────────┬─────────────┬────────┬──────────┬──────────────────┬──────────┐
│ Backend │ WaterLily │ Julia │ Precision │ Allocations │ GC [%] │ Time [s] │ Cost [ns/DOF/dt] │ Speed-up │
├────────────┼───────────┼────────┼───────────┼─────────────┼────────┼──────────┼──────────────────┼──────────┤
│ CPUx01 │ master │ 1.11.5 │ Float32 │ 8307 │ 0.00 │ 25.63 │ 244.38 │ 1.00 │
│ CPUx01 │ master │ 1.12.0 │ Float32 │ 7307 │ 0.00 │ 26.46 │ 252.35 │ 0.97 │
│ CPUx02 │ master │ 1.11.5 │ Float32 │ 4229004 │ 0.00 │ 28.61 │ 272.88 │ 0.90 │
│ CPUx02 │ master │ 1.12.0 │ Float32 │ 4136532 │ 0.00 │ 30.73 │ 293.07 │ 0.83 │
│ CPUx04 │ master │ 1.11.5 │ Float32 │ 5860776 │ 0.17 │ 17.27 │ 164.69 │ 1.48 │
│ CPUx04 │ master │ 1.12.0 │ Float32 │ 1592091668 │ 10.72 │ 26.24 │ 250.23 │ 0.98 │
│ GPU-NVIDIA │ master │ 1.11.5 │ Float32 │ 7780323 │ 0.48 │ 3.26 │ 31.14 │ 7.85 │
│ GPU-NVIDIA │ master │ 1.12.0 │ Float32 │ 6813568 │ 0.51 │ 3.24 │ 30.92 │ 7.90 │
└────────────┴───────────┴────────┴───────────┴─────────────┴────────┴──────────┴──────────────────┴──────────┘
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the WaterLily-Benchmarks benchmark.sh command and compare.jl using Julia 1.11.5 and 1.12.0, with attention to the CPUx04 results. Compare allocations, GC time, and runtime across the tgv and jelly cases; done means the CPU multi-threading regression is explained and the affected benchmarks no longer show the reported Julia 1.12.0 degradation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100