trixi-framework / trixi-framework/Trixi.jl
Some elixirs fail with `StackOverflowError` on multiple threads
- Dominant language
- Julia
- Stars
- 731
- Forks
- 166
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 25
Description
For example:
```julia
julia> include("examples/p4est_3d_dgsem/elixir_euler_circular_wind_nonconforming.jl")
```
```
████████╗██████╗ ██╗██╗ ██╗██╗
╚══██╔══╝██╔══██╗██║╚██╗██╔╝██║
██║ ██████╔╝██║ ╚███╔╝ ██║
██║ ██╔══██╗██║ ██╔██╗ ██║
██║ ██║ ██║██║██╔╝ ██╗██║
╚═╝ ╚═╝ ╚═╝╚═╝╚═╝ ╚═╝╚═╝
┌──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ SemidiscretizationHyperbolic │
│ ════════════════════════════ │
│ #spatial dimensions: ………………………… 3 │
│ mesh: ………………………………………………………………… P4estMesh{3, Float64} │
│ equations: …………………………………………………… CompressibleEulerEquations3D │
│ initial condition: ……………………………… initial_condition_circular_wind │
│ boundary conditions: ………………………… 2 │
│ │ inside: ……………………………………………………… typeof(boundary_condition_slip_wall) │
│ │ outside: …………………………………………………… typeof(boundary_condition_slip_wall) │
│ source terms: …………………………………………… source_terms_circular_wind │
│ solver: …………………………………………………………… DG │
│ total #DOFs: ……………………………………………… 54000 │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘
[...]
────────────────────────────────────────────────────────────────────────────────────────────────────
Simulation running 'CompressibleEulerEquations3D' with DGSEM(polydeg=4)
────────────────────────────────────────────────────────────────────────────────────────────────────
#timesteps: 0 run time: 7.00000000e-07 s
Δt: 0.00000000e+00 time/DOF/rhs!: NaN s
sim. time: 0.00000000e+00
#DOF: 54000
#elements: 432
└── level 0: 432
Variable: rho rho_v1 rho_v2 rho_v3 rho_e
L2 error: 2.20459321e-16 1.12109309e-15 1.12171013e-15 0.00000000e+00 1.04810638e-05
Linf error: 5.55111512e-16 8.88178420e-15 8.88178420e-15 0.00000000e+00 5.84162772e-05
∑∂S/∂U ⋅ Uₜ : -2.99530351e-15
────────────────────────────────────────────────────────────────────────────────────────────────────
ERROR: LoadError: StackOverflowError:
Stacktrace:
[1] materialize!(dest::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, bc::Base.Broadcast.Broadcasted{StrideArrays.CartesianStyle{Tuple{Int64}, 1}, Nothing, typeof(DiffEqBase.calculate_residuals), Tuple{StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, Float64, Float64, Base.RefValue{typeof(DiffEqBase.ODE_DEFAULT_NORM)}, Float64}})
@ StrideArrays ~/.julia/packages/StrideArrays/RDSui/src/broadcast.jl:185
[2] macro expansion
@ ~/.julia/packages/LoopVectorization/DDH6Z/src/broadcast.jl:530 [inlined]
[3] vmaterialize!(dest::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, bc::Base.Broadcast.Broadcasted{StrideArrays.CartesianStyle{Tuple{Int64}, 1}, Nothing, typeof(DiffEqBase.calculate_residuals), Tuple{StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, Float64, Float64, Base.RefValue{typeof(DiffEqBase.ODE_DEFAULT_NORM)}, Float64}}, #unused#::Val{:StrideArrays}, #unused#::Val{(true, 0, 0, 0, true, 4, 32, 15, 64, 0x0000000000000001, 0, true)}, #unused#::Val{((true,), (true,), (true,), (), (), (), ())})
@ LoopVectorization ~/.julia/packages/LoopVectorization/DDH6Z/src/broadcast.jl:530
[4] vmaterialize!(dest::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, bc::Base.Broadcast.Broadcasted{StrideArrays.CartesianStyle{Tuple{Int64}, 1}, Nothing, typeof(DiffEqBase.calculate_residuals), Tuple{StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, Float64, Float64, Base.RefValue{typeof(DiffEqBase.ODE_DEFAULT_NORM)}, Float64}}, #unused#::Val{:StrideArrays}, #unused#::Val{(true, 0, 0, 0, true, 4, 32, 15, 64, 0x0000000000000001, 0, true)})
@ LoopVectorization ~/.julia/packages/LoopVectorization/DDH6Z/src/broadcast.jl:664
[5] _materialize!(dest::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, bc::Base.Broadcast.Broadcasted{StrideArrays.CartesianStyle{Tuple{Int64}, 1}, Nothing, typeof(DiffEqBase.calculate_residuals), Tuple{StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, Float64, Float64, Base.RefValue{typeof(DiffEqBase.ODE_DEFAULT_NORM)}, Float64}}, #unused#::Val{(true, 0, 0, 0, true, 4, 32, 15, 64, 0x0000000000000001, 0, true)})
@ StrideArrays ~/.julia/packages/StrideArrays/RDSui/src/broadcast.jl:178
--- the last 5 lines are repeated 19994 more times ---
[99976] materialize!(dest::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, bc::Base.Broadcast.Broadcasted{StrideArrays.CartesianStyle{Tuple{Int64}, 1}, Nothing, typeof(DiffEqBase.calculate_residuals), Tuple{StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, Float64, Float64, Base.RefValue{typeof(DiffEqBase.ODE_DEFAULT_NORM)}, Float64}})
@ StrideArrays ~/.julia/packages/StrideArrays/RDSui/src/broadcast.jl:185
[99977] macro expansion
@ ~/.julia/packages/LoopVectorization/DDH6Z/src/broadcast.jl:530 [inlined]
[99978] vmaterialize!(dest::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, bc::Base.Broadcast.Broadcasted{StrideArrays.CartesianStyle{Tuple{Int64}, 1}, Nothing, typeof(DiffEqBase.calculate_residuals), Tuple{StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, Float64, Float64, Base.RefValue{typeof(DiffEqBase.ODE_DEFAULT_NORM)}, Float64}}, #unused#::Val{:StrideArrays}, #unused#::Val{(true, 0, 0, 0, true, 4, 32, 15, 64, 0x0000000000000001, 0, true)}, #unused#::Val{((true,), (true,), (true,), (), (), (), ())})
@ LoopVectorization ~/.julia/packages/LoopVectorization/DDH6Z/src/broadcast.jl:530
[99979] vmaterialize!(dest::StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, bc::Base.Broadcast.Broadcasted{StrideArrays.CartesianStyle{Tuple{Int64}, 1}, Nothing, typeof(DiffEqBase.calculate_residuals), Tuple{StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, StrideArraysCore.PtrArray{Tuple{Int64}, (true,), Float64, 1, 1, 0, (1,), Tuple{Static.StaticInt{8}}, Tuple{Static.StaticInt{1}}}, Float64, Float64, Base.RefValue{typeof(DiffEqBase.ODE_DEFAULT_NORM)}, Float64}}, #unused#::Val{:StrideArrays}, #unused#::Val{(true, 0, 0, 0, true, 4, 32, 15, 64, 0x0000000000000001, 0, true)})
@ LoopVectorization ~/.julia/packages/LoopVectorization/DDH6Z/src/broadcast.jl:664
in expression starting at /mnt/ssd/home/efaulhab/git/Trixi.jl/examples/p4est_3d_dgsem/elixir_euler_circular_wind_nonconforming.jl:161
```
This affects all elixirs using `RDPK3SpFSAL49` with threading enabled (`RDPK3SpFSAL49(thread=OrdinaryDiffEq.True())`) and is caused by https://github.com/JuliaSIMD/StrideArrays.jl/issues/62.
Contributor guide
Research direction
Start with examples/p4est_3d_dgsem/elixir_euler_circular_wind_nonconforming.jl at the reported line 161, and reproduce the failure using RDPK3SpFSAL49(thread=OrdinaryDiffEq.True()). Compare the affected elixirs that use this threaded solver and the linked StrideArrays.jl issue; done means those elixirs no longer produce StackOverflowError with threading enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100