JuliaCI / JuliaCI/BenchmarkTools.jl
Unrealistic values of same tests after repeated executions
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 684
- Forks
- 107
- PR merge metrics
- No merged PRs in 30d
Description
The following simple tests behave differently in a second execution.
using BenchmarkTools
using LinearAlgebra
a = rand(300,300); u = triu(rand(300,300));
@btime u*a; # multiplication without exploiting upper triangular shape
@btime UpperTriangular(u)*a; # multiplication with exploitation of upper triangular shape
@btime lmul!(UpperTriangular(u),a); # in-place multiplication
@btime u*a; # multiplication without exploiting upper triangular shape
@btime UpperTriangular(u)*a; # multiplication with exploitation of upper triangular shape
@btime lmul!(UpperTriangular(u),a); # in-place multiplication
These are the results folowing a new start of Julia (other executions produce different figures):
julia> @btime u*a; # multiplication without exploiting upper triangular shape
281.900 μs (2 allocations: 703.17 KiB)
julia> @btime UpperTriangular(u)*a; # multiplication with exploitation of upper triangular shape
167.200 μs (3 allocations: 703.19 KiB)
julia> @btime lmul!(UpperTriangular(u),view(a,:,:)); # in-place multiplication
845.800 μs (5 allocations: 176 bytes)
julia> @btime u*a; # multiplication without exploiting upper triangular shape
3.432 ms (2 allocations: 703.17 KiB)
julia> @btime UpperTriangular(u)*a; # multiplication with exploitation of upper triangular shape
3.011 ms (3 allocations: 703.19 KiB)
julia> @btime lmul!(UpperTriangular(u),view(a,:,:)); # in-place multiplication
3.045 ms (5 allocations: 176 bytes)
Some values show a more than 10-times slow down. Is any explanation for this behaviour?
julia> versioninfo()
Julia Version 1.8.3
Commit 0434deb161 (2022-11-14 20:14 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: 16 × Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
Threads: 1 on 16 virtual cores
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported benchmark sequence from the issue on Julia 1.8.3 under Windows, including the repeated calls to u*a, UpperTriangular(u)*a, and lmul!. Compare the first and second results, then determine and document the cause of the reported slowdown; done means the differing timings have a confirmed explanation or a narrowly defined fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100