JuliaCI / JuliaCI/BaseBenchmarks.jl
tame benchmark time
Open
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 44
- Forks
- 45
- PR merge metrics
- No merged PRs in 30d
Description
The tests take too long, here's why:
$ grep '^[^ ]' 093b2a6ea943f4c70fef4742453e73dd1aba255c_primary.out
RUNNING BENCHMARKS...
(1/20) benchmarking "shootout"...
done (took 70.756991683 seconds)
(2/20) benchmarking "string"...
done (took 90.13968044 seconds)
(3/20) benchmarking "linalg"...
done (took 833.003473297 seconds)
(4/20) benchmarking "parallel"...
done (took 17.074202133 seconds)
(5/20) benchmarking "find"...
done (took 139.282557616 seconds)
(6/20) benchmarking "tuple"...
done (took 107.72978849 seconds)
(7/20) benchmarking "dates"...
done (took 112.767580891 seconds)
(8/20) benchmarking "micro"...
done (took 43.087218279 seconds)
(9/20) benchmarking "io"...
done (took 61.65084442 seconds)
(10/20) benchmarking "scalar"... # not unreasonable tests, but takes 2 seconds each
done (took 2877.452812717 seconds)
(11/20) benchmarking "sparse"... # "matmul" has over 200 combinations, probably could be more "sparse"
done (took 1278.73521643 seconds)
(12/20) benchmarking "broadcast"...
done (took 104.842238879 seconds)
(13/20) benchmarking "union"... # 341 tests may be reasonable, but taking 3 seconds each
done (took 1192.683332499 seconds)
(14/20) benchmarking "simd"...
done (took 337.777793678 seconds)
(15/20) benchmarking "random"...
done (took 619.551763952 seconds)
(16/20) benchmarking "problem"...
done (took 172.609170893 seconds)
(17/20) benchmarking "array"... # 608 "index" tests may be a bit much, and take 4 seconds each
done (took 2795.986356153 seconds)
(18/20) benchmarking "misc"...
done (took 158.855686545 seconds)
(19/20) benchmarking "sort"...
done (took 648.501236383 seconds)
(20/20) benchmarking "collection"...
done (took 916.806169471 seconds)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the benchmark entry point that produces the 20 runs shown in the issue and reproduce the reported timings. Focus first on the scalar, sparse, union, array, and other longest-running benchmarks identified in the output. Done means reducing excessive benchmark time while preserving the benchmark coverage and confirming the full run completes successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100