JuliaLang / JuliaLang/LinearAlgebra.jl

Extremely poor performance of auto global variable distribution with Diagonals

Open
#679 3 comments 0 reactions 0 assignees View on GitHub
parallelism
Dominant language
Julia
Stars
77
Forks
65
Avg merge
3d 23h
Merged PRs (30d)
10

Description

A MWE (1.2 or 1.3RC4):

```julia
using LinearAlgebra
using Distributed
using BenchmarkTools
addprocs(1)
n = 128^2
d = rand(Float32,n)
D = Diagonal(d)

@belapsed @fetch D # ~1 second
```

Looks like its the combination of the wrapping in a `Diagonal` and auto global variable distribution that triggers it, because both of these, which should be functionally equivalent, are fine:

```julia
@belapsed @fetch d # ~10^-3 second
@belapsed let D=D; @fetch D; end # 10^-3 second
```

I'm guessing some of that global variable distribution code is treating Diagonals as dense matrices but should instead be specialized?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the Julia MWE with Distributed, LinearAlgebra, Diagonal, and @fetch, then compare timings for D, d, and the let-bound D cases. Trace the auto-global distribution path entered by @fetch and inspect how Diagonal is handled versus a dense array. Done means the Diagonal case no longer incurs the reported ~1-second overhead while preserving the existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.