CliMA / CliMA/ClimaCore.jl

MatrixFields invalidates 3660 methods

Open
#1,812 0 comments 0 reactions 0 assignees View on GitHub
bug Invalidations
Dominant language
Julia
Stars
117
Forks
19
Avg merge
3d 4h
Merged PRs (30d)
41

Description

The invalidations job in [ClimaAtmos.jl](https://github.com/CliMA/ClimaAtmos.jl), where we use SnoopCompile's [invalidations script](https://timholy.github.io/SnoopCompile.jl/dev/tutorial/#Cut-to-the-Chase:-A-copy-paste-analysis-of-invalidations), shows that ClimaCore's `broadcasted`:

```
[ Info: 4296 methods invalidated for 178 functions (showing 10 functions)
┌─────────────────────────────────────────────────────────┬───────────────┬───────────────┬─────────────────┐
│ : │ Function Name │ Invalidations │ Invalidations % │
│ │ │ │ (xᵢ/∑x) │
├─────────────────────────────────────────────────────────┼───────────────┼───────────────┼─────────────────┤
│ DiskArrays/bZBJE/src/zip.jl:83 │ zip │ 3247 │ 14 │
│ DiskArrays/bZBJE/src/zip.jl:70 │ zip │ 3247 │ 14 │
│ DiskArrays/bZBJE/src/zip.jl:83 │ zip │ 3247 │ 14 │
│ FillArrays/eOEVm/src/fillbroadcast.jl:78 │ broadcasted │ 1220 │ 5 │
│ CommonDataModel/G3moc/src/groupby.jl:267 │ broadcasted │ 1220 │ 5 │
│ ClimaCore/ANgUC/src/MatrixFields/field_name_dict.jl:331 │ broadcasted │ 1220 │ 5 │
│ ClimaCore/ANgUC/src/MatrixFields/field_name_dict.jl:475 │ broadcasted │ 1220 │ 5 │
│ ClimaCore/ANgUC/src/MatrixFields/lazy_operators.jl:44 │ broadcasted │ 1220 │ 5 │
│ GeometryBasics/ebXl0/src/offsetintegers.jl:40 │ convert │ 834 │ 4 │
│ DiskArrays/bZBJE/src/generator.jl:53 │ Generator │ 439 │ 2 │
└─────────────────────────────────────────────────────────┴───────────────┴───────────────┴─────────────────┘
```

so
```julia
Base.Broadcast.broadcasted(
::FieldNameDictStyle,
f::F, # This should be restricted to a Function to avoid a method ambiguity.
args...,
) where {F <: Function} = field_matrix_broadcast_error(f, args...)
...
Base.Broadcast.broadcasted(
::FieldNameDictStyle,
f::F,
arg::FieldVectorStyleType,
) where {F <: Function} =
Base.Broadcast.broadcasted(f, convert_to_field_name_dict(arg))
...
Base.Broadcast.broadcasted(::LazyOperatorStyle, f::F, args...) where {F} =
LazyOperatorBroadcasted(f, args)...
```
are invalidating 3660 methods.

This seems to suggest that the invalidation measurements in the comments of `src/MatrixFields/lazy_operators.jl` were not captured in the same way (invalidations depends on the workload).

cc @dennisYatunin

Contributor guide

No contributing guide indexed for this repository

Research direction

Run the invalidations job in ClimaAtmos using the SnoopCompile invalidations script, then inspect src/MatrixFields/lazy_operators.jl and src/MatrixFields/field_name_dict.jl at the reported broadcasted methods. Compare the measured invalidations with the comments in lazy_operators.jl; done requires identifying the source of the 3660 invalidations and recording a concrete resolution or follow-up.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.