JuliaArrays / JuliaArrays/FillArrays.jl

Benchmark suite

Open
#51 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
187
Forks
41
Avg merge
17h 29m
Merged PRs (30d)
1

Description

What about adding benchmarks to the repo ? Here is the beginning of a benchmark suite.

I was unable to find any convenient tools for formatting and organizing reports. Whats more, after poking around in a few packages, it seems that these things tend to fall behind the rest of the code.

But, it helps me keep track of what has been done. For instance, here are the relevant benchmarks before the pending PR:

julia> println.(result["Eye"]["identities"]);
Pair{Any,Any}(("inv", "Int64", "n=1000"), Trial(960.983 ns))
Pair{Any,Any}(("inv", "Float64", "n=10"), Trial(53.983 ns))
Pair{Any,Any}(("LinearAlgebra.triu", "Int64", "n=10"), Trial(55.983 ns))
Pair{Any,Any}(("permutedims", "Float64", "n=1000"), Trial(6.776 ms))
Pair{Any,Any}(("inv", "Int64", "n=10"), Trial(52.983 ns))
Pair{Any,Any}(("LinearAlgebra.tril", "Float64", "n=10"), Trial(60.983 ns))
Pair{Any,Any}(("LinearAlgebra.tril", "Float64", "n=1000"), Trial(718.983 ns))
Pair{Any,Any}(("LinearAlgebra.triu", "Float64", "n=10"), Trial(54.983 ns))
Pair{Any,Any}(("inv", "Float64", "n=1000"), Trial(970.983 ns))
Pair{Any,Any}(("LinearAlgebra.tril", "Int64", "n=10"), Trial(55.983 ns))
Pair{Any,Any}(("permutedims", "Int64", "n=10"), Trial(1.538 μs))
Pair{Any,Any}(("permutedims", "Float64", "n=10"), Trial(1.543 μs))
Pair{Any,Any}(("permutedims", "Int64", "n=1000"), Trial(6.678 ms))
Pair{Any,Any}(("LinearAlgebra.triu", "Int64", "n=1000"), Trial(619.983 ns))
Pair{Any,Any}(("LinearAlgebra.triu", "Float64", "n=1000"), Trial(679.983 ns))
Pair{Any,Any}(("LinearAlgebra.tril", "Int64", "n=1000"), Trial(705.983 ns))

Here are the same benchmarks after the PR:

julia> println.(result["Eye"]["identities"]);
Pair{Any,Any}(("inv", "Int64", "n=1000"), Trial(0.001 ns))
Pair{Any,Any}(("inv", "Float64", "n=10"), Trial(0.001 ns))
Pair{Any,Any}(("LinearAlgebra.triu", "Int64", "n=10"), Trial(0.001 ns))
Pair{Any,Any}(("permutedims", "Float64", "n=1000"), Trial(0.001 ns))
Pair{Any,Any}(("inv", "Int64", "n=10"), Trial(0.001 ns))
Pair{Any,Any}(("LinearAlgebra.tril", "Float64", "n=10"), Trial(0.001 ns))
Pair{Any,Any}(("LinearAlgebra.tril", "Float64", "n=1000"), Trial(0.001 ns))
Pair{Any,Any}(("LinearAlgebra.triu", "Float64", "n=10"), Trial(0.001 ns))
Pair{Any,Any}(("inv", "Float64", "n=1000"), Trial(0.001 ns))
Pair{Any,Any}(("LinearAlgebra.tril", "Int64", "n=10"), Trial(0.001 ns))
Pair{Any,Any}(("permutedims", "Int64", "n=10"), Trial(0.001 ns))
Pair{Any,Any}(("permutedims", "Float64", "n=10"), Trial(0.001 ns))
Pair{Any,Any}(("permutedims", "Int64", "n=1000"), Trial(0.001 ns))
Pair{Any,Any}(("LinearAlgebra.triu", "Int64", "n=1000"), Trial(0.001 ns))
Pair{Any,Any}(("LinearAlgebra.triu", "Float64", "n=1000"), Trial(0.001 ns))
Pair{Any,Any}(("LinearAlgebra.tril", "Int64", "n=1000"), Trial(0.001 ns))

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the proposed benchmark/benchmark/FillArraysBenchmarks.jl file and the benchmark results shown in this issue. Determine how benchmarks should be organized and reports formatted; done means the repository has a usable benchmark suite that tracks the relevant operations and produces organized results.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.