JuliaArrays / JuliaArrays/StaticArrays.jl

Matrix multiplication logic poor for ForwardDiff.Dual

Open
#513 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

linear-algebra performance
Dominant language
Julia
Stars
844
Forks
159
Avg merge
3d 21h
Merged PRs (30d)
3

Description

StaticArrays has heuristics that determine what code to make to multiply matrices. There seems to have a heuristic for BlasFloat and one for everything else (i.e.Any). The present Any heuristic makes bad choices for Dual. I think that it would be straightforward to create a better heuristic for ForwardDiff.Dual by including the number of partials in the heuristic. The obvious issue with this is that it would require ForwardDiff to be a dependency to StaticArrays. Is there a good way to get this performance issue fixed?

using BenchmarkTools
using ForwardDiff
using StaticArrays

Type_Dual = ForwardDiff.Dual{Float64,Float64,26}

A = rand(SMatrix{4,4,Type_Dual,16})
B = rand(SMatrix{4,4,Type_Dual,16})

@btime $A * $B  # DEFAULT
# 1.376 μs (0 allocations: 0 bytes)

@btime StaticArrays.mul_loop($(Size(A)),$(Size(B)),$A,$B)
# 614.142 ns (0 allocations: 0 bytes)

@btime StaticArrays.mul_unrolled_chunks($(Size(A)),$(Size(B)),$A,$B)
# 688.962 ns (0 allocations: 0 bytes)

@btime StaticArrays.mul_unrolled($(Size(A)),$(Size(B)),$A,$B)
# 1.382 μs (0 allocations: 0 bytes)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported Julia benchmark with ForwardDiff.Dual and compare the default matrix multiplication path against StaticArrays.mul_loop, mul_unrolled_chunks, and mul_unrolled. Investigate how the current heuristic selects among these entry points and how the number of Dual partials could be considered without an unwanted dependency. Done means the reported Dual multiplication uses an improved path with benchmark coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.