trixi-framework / trixi-framework/Trixi.jl

Wild allocations of `RDPK3SpFSAL49` for `DGMulti`

Open
#2,832 7 comments 0 reactions 0 assignees View on GitHub
bug upstream
Dominant language
Julia
Stars
731
Forks
167
Avg merge
2d 18h
Merged PRs (30d)
25

Description

This is the summary callback for

https://github.com/trixi-framework/Trixi.jl/blob/main/examples/dgmulti_2d/elixir_euler_kelvin_helmholtz_instability.jl

with
```julia
sol = solve(ode, RDPK3SpFSAL49();
adaptive = false, dt = estimate_dt(mesh, dg),
ode_default_options()..., callback = callbacks);
```

```julia
────────────────────────────────────────────────────────────────────────────────────
Trixi.jl Time Allocations
─────────────────────── ────────────────────────
Tot / % measured: 54.6s / 3.1% 49.8GiB / 0.0%

Section ncalls time %tot avg alloc %tot avg
────────────────────────────────────────────────────────────────────────────────────
rhs! 1.74k 1.67s 99.1% 962μs 5.88KiB 0.0% 3.46B
volume integral 1.74k 1.20s 71.1% 691μs 0.00B 0.0% 0.00B
interface flux 1.74k 299ms 17.7% 172μs 0.00B 0.0% 0.00B
surface integral 1.74k 54.1ms 3.2% 31.1μs 0.00B 0.0% 0.00B
prolong2interfaces 1.74k 51.6ms 3.1% 29.7μs 0.00B 0.0% 0.00B
reset ∂u/∂t 1.74k 39.5ms 2.3% 22.7μs 0.00B 0.0% 0.00B
Jacobian 1.74k 23.6ms 1.4% 13.6μs 0.00B 0.0% 0.00B
~rhs!~ 1.74k 4.22ms 0.2% 2.43μs 5.88KiB 0.0% 3.46B
boundary flux 1.74k 55.3μs 0.0% 31.8ns 0.00B 0.0% 0.00B
source terms 1.74k 37.2μs 0.0% 21.4ns 0.00B 0.0% 0.00B
I/O 4 10.3ms 0.6% 2.58ms 16.5MiB 91.4% 4.13MiB
save solution 3 10.3ms 0.6% 3.43ms 16.5MiB 91.4% 5.51MiB
~I/O~ 4 6.21μs 0.0% 1.55μs 4.44KiB 0.0% 1.11KiB
get node variables 3 580ns 0.0% 193ns 0.00B 0.0% 0.00B
save mesh 3 375ns 0.0% 125ns 0.00B 0.0% 0.00B
get element variables 3 66.0ns 0.0% 22.0ns 0.00B 0.0% 0.00B
analyze solution 3 5.39ms 0.3% 1.80ms 1.55MiB 8.6% 530KiB
────────────────────────────────────────────────────────────────────────────────────
```

For fixed timestep with

```julia
sol = solve(ode, CarpenterKennedy2N54(williamson_condition = false);
dt = estimate_dt(mesh, dg), ode_default_options()..., callback = callbacks);
```

```julia
────────────────────────────────────────────────────────────────────────────────────
Trixi.jl Time Allocations
─────────────────────── ────────────────────────
Tot / % measured: 919ms / 86.7% 28.1MiB / 64.4%

Section ncalls time %tot avg alloc %tot avg
────────────────────────────────────────────────────────────────────────────────────
rhs! 966 780ms 97.9% 807μs 5.88KiB 0.0% 6.23B
volume integral 966 577ms 72.3% 597μs 0.00B 0.0% 0.00B
interface flux 966 144ms 18.1% 149μs 0.00B 0.0% 0.00B
surface integral 966 22.4ms 2.8% 23.1μs 0.00B 0.0% 0.00B
prolong2interfaces 966 14.6ms 1.8% 15.1μs 0.00B 0.0% 0.00B
Jacobian 966 11.3ms 1.4% 11.7μs 0.00B 0.0% 0.00B
reset ∂u/∂t 966 9.80ms 1.2% 10.2μs 0.00B 0.0% 0.00B
~rhs!~ 966 913μs 0.1% 945ns 5.88KiB 0.0% 6.23B
boundary flux 966 25.7μs 0.0% 26.6ns 0.00B 0.0% 0.00B
source terms 966 15.3μs 0.0% 15.9ns 0.00B 0.0% 0.00B
I/O 4 12.1ms 1.5% 3.03ms 16.6MiB 91.4% 4.14MiB
save solution 3 11.3ms 1.4% 3.76ms 16.5MiB 91.3% 5.51MiB
~I/O~ 4 828μs 0.1% 207μs 28.1KiB 0.2% 7.02KiB
save mesh 3 625ns 0.0% 208ns 0.00B 0.0% 0.00B
get node variables 3 531ns 0.0% 177ns 0.00B 0.0% 0.00B
get element variables 3 64.0ns 0.0% 21.3ns 0.00B 0.0% 0.00B
analyze solution 3 4.96ms 0.6% 1.65ms 1.55MiB 8.6% 529KiB
────────────────────────────────────────────────────────────────────────────────────
```

DGSEM (https://github.com/trixi-framework/Trixi.jl/blob/main/examples/tree_2d_dgsem/elixir_euler_kelvin_helmholtz_instability.jl) with

```julia
sol = solve(ode, RDPK3SpFSAL49();
adaptive = false, dt = 1.0,
ode_default_options()..., callback = callbacks);
```

```julia
────────────────────────────────────────────────────────────────────────────────────
Trixi.jl Time Allocations
─────────────────────── ────────────────────────
Tot / % measured: 9.28s / 94.1% 55.1MiB / 91.7%

Section ncalls time %tot avg alloc %tot avg
────────────────────────────────────────────────────────────────────────────────────
rhs! 5.66k 8.61s 98.6% 1.52ms 10.1KiB 0.0% 1.82B
volume integral 5.66k 7.10s 81.3% 1.25ms 752B 0.0% 0.13B
~volume integral~ 5.66k 6.74s 77.3% 1.19ms 752B 0.0% 0.13B
blending factors 5.66k 354ms 4.1% 62.6μs 0.00B 0.0% 0.00B
interface flux 5.66k 735ms 8.4% 130μs 0.00B 0.0% 0.00B
prolong2interfaces 5.66k 359ms 4.1% 63.4μs 0.00B 0.0% 0.00B
surface integral 5.66k 286ms 3.3% 50.5μs 0.00B 0.0% 0.00B
reset ∂u/∂t 5.66k 64.0ms 0.7% 11.3μs 0.00B 0.0% 0.00B
Jacobian 5.66k 52.6ms 0.6% 9.29μs 0.00B 0.0% 0.00B
~rhs!~ 5.66k 9.55ms 0.1% 1.69μs 9.33KiB 0.0% 1.69B
prolong2mortars 5.66k 646μs 0.0% 114ns 0.00B 0.0% 0.00B
prolong2boundaries 5.66k 624μs 0.0% 110ns 0.00B 0.0% 0.00B
mortar flux 5.66k 392μs 0.0% 69.3ns 0.00B 0.0% 0.00B
boundary flux 5.66k 152μs 0.0% 26.9ns 0.00B 0.0% 0.00B
source terms 5.66k 113μs 0.0% 19.9ns 0.00B 0.0% 0.00B
I/O 34 51.9ms 0.6% 1.53ms 50.0MiB 98.9% 1.47MiB
save solution 33 49.8ms 0.6% 1.51ms 50.0MiB 98.8% 1.51MiB
get element variables 33 2.00ms 0.0% 60.5μs 0.00B 0.0% 0.00B
~I/O~ 34 100μs 0.0% 2.95μs 19.4KiB 0.0% 585B
save mesh 33 14.7μs 0.0% 447ns 0.00B 0.0% 0.00B
get node variables 33 11.3μs 0.0% 343ns 0.00B 0.0% 0.00B
calculate dt 630 48.0ms 0.5% 76.1μs 0.00B 0.0% 0.00B
analyze solution 8 22.9ms 0.3% 2.86ms 571KiB 1.1% 71.3KiB
────────────────────────────────────────────────────────────────────────────────────
```

I would guess something is going horribly wrong with the `DGMulti` types. Must be one of the recent `OrdinaryDiffEq` releases.

Contributor guide

Open the contributing guide

Research direction

Start with examples/dgmulti_2d/elixir_euler_kelvin_helmholtz_instability.jl and examples/tree_2d_dgsem/elixir_euler_kelvin_helmholtz_instability.jl, reproducing the reported solve calls and allocation profiles. Compare the DGMulti and DGSEM results, then inspect the RDPK3SpFSAL49 and OrdinaryDiffEq paths implicated by the report. Done means identifying the source of the unexpected DGMulti allocations and demonstrating corrected allocation behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.