JuliaCI/BenchmarkTools.jl

View on GitHub

A benchmarking framework for the Julia language

Stars: 684
Forks: 107
Open beginner issues: 0
Indexed issues: 72

Dominant language: Julia
License: No license data
Last GitHub push: Aug 7, 2026
Latest indexed: Sep 18, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

72 open issues indexed

Overloading more features from Statistics Open

Difficulty 3/5 1-2 days Newbie friendliness 50/100

JuliaCI/BenchmarkTools.jl#434 · Opened Sep 2, 2026
Deprecation Warnings under julia1.12 Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

JuliaCI/BenchmarkTools.jl#412 · Opened Mar 9, 2026
Repeated `@btime` with different setup behaves bad Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#403 · 5 comments · 1 reaction · Opened Oct 9, 2025
Restructuring a benchmark group breaks loading tuning parameters and judging Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#396 · Opened Jun 18, 2025
The metrics for the mean and standard deviation of (percentage-wise) GC are not consistent with each other Open

Difficulty 2/5 1-3 hours Newbie friendliness 58/100

JuliaCI/BenchmarkTools.jl#394 · Opened May 18, 2025
Detect warmup in benchmarked code Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#393 · Opened May 14, 2025
Why it is not possible to use for loops inside the setup block? Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

JuliaCI/BenchmarkTools.jl#392 · Opened Apr 25, 2025
Factor out BenchmarkGroup Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

JuliaCI/BenchmarkTools.jl#390 · 1 comment · Opened Jan 9, 2025
Benchmarked function is run one more time than specified Open

Difficulty 3/5 1-2 days Newbie friendliness 38/100

JuliaCI/BenchmarkTools.jl#382 · 1 comment · Opened Dec 18, 2024
Unable to parse argument destructuring syntax Open

Difficulty 3/5 1-2 days Newbie friendliness 52/100

JuliaCI/BenchmarkTools.jl#379 · 1 reaction · Opened Nov 16, 2024
LinuxPerf, @profile, and other experiments Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

JuliaCI/BenchmarkTools.jl#377 · 6 comments · Opened Nov 1, 2024
Very slow printing of CuArray in benchmark result Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

JuliaCI/BenchmarkTools.jl#376 · 1 comment · Opened Oct 30, 2024
Default parameter option `evals_set` is not documented Open

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

JuliaCI/BenchmarkTools.jl#374 · 2 comments · Opened Sep 24, 2024
Geometric mean as a measure Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#371 · 1 comment · Opened Aug 20, 2024
`@btime` reports wrong allocation Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

JuliaCI/BenchmarkTools.jl#360 · 1 comment · Opened Mar 11, 2024
@benchmark creates new symbol for each interpolation of a symbol Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#329 · 3 comments · Opened Aug 29, 2023
Run docs examples during CI instead of copypasting outputs Open

documentation

Difficulty 4/5 3-5 days Newbie friendliness 25/100

JuliaCI/BenchmarkTools.jl#319 · Opened Jun 14, 2023
Memory estimate? Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 30/100

JuliaCI/BenchmarkTools.jl#311 · 3 reactions · Opened Jun 3, 2023
Document units of memory and time properties of Trial type Open

documentation

Difficulty 1/5 Under an hour Newbie friendliness 55/100

JuliaCI/BenchmarkTools.jl#307 · 1 reaction · Opened May 2, 2023
Unrealistic values of same tests after repeated executions Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 25/100

JuliaCI/BenchmarkTools.jl#306 · 8 comments · Opened May 1, 2023
@btime in local scope throws UndefVarError Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#287 · 3 comments · Opened Jul 11, 2022
Feature idea: Check which function is faster Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

JuliaCI/BenchmarkTools.jl#283 · 4 comments · 2 reactions · Opened Apr 26, 2022
No output when running @benchmark from script file Open

documentation

Difficulty 3/5 1-2 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#281 · 3 comments · 1 reaction · Opened Apr 20, 2022
Function undefined for @btime when using for loop Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 45/100

JuliaCI/BenchmarkTools.jl#280 · 2 comments · Opened Mar 13, 2022
Benchmarks only track minimum number of allocations and memory usage Open

documentation

Difficulty 4/5 3-5 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#277 · 1 comment · 1 reaction · Opened Feb 19, 2022
Feature suggestion: @ballocs Open

enhancement

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

JuliaCI/BenchmarkTools.jl#272 · 1 comment · Opened Dec 19, 2021
Noticeable performance penalty after "interpolated" benchmarks on MacOS Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#263 · Opened Nov 15, 2021
common histogram bounds when displaying multiple benchmarks Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 32/100

JuliaCI/BenchmarkTools.jl#256 · 2 comments · 1 reaction · Opened Sep 16, 2021
Improper axis limits in histograms Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 45/100

JuliaCI/BenchmarkTools.jl#253 · 6 comments · Opened Sep 13, 2021
Trial display assumes the max GC comes from the sample with the max time Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#248 · 1 comment · Opened Aug 16, 2021
ERROR: DomainError with -43.0: log will only return a complex result if called with a complex argument. Try log(Complex(x)). Open

bug

JuliaCI/BenchmarkTools.jl#245 · 5 comments · 1 reaction · 1 assignee · Opened Aug 5, 2021
Provide entry point for color customization Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 42/100

JuliaCI/BenchmarkTools.jl#244 · 1 comment · Opened Aug 5, 2021
Feature Request: `@benchmark f() g()` Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#239 · 7 comments · 2 reactions · Opened Jul 27, 2021
Miss-aligned text (likely issue with unicode?) Open

enhancement

JuliaCI/BenchmarkTools.jl#238 · 5 comments · 1 assignee · Opened Jul 21, 2021
log(frequency) in histograms confusing Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 45/100

JuliaCI/BenchmarkTools.jl#237 · 16 comments · 5 reactions · Opened Jul 16, 2021
More copy-able result for `@benchmark` Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 45/100

JuliaCI/BenchmarkTools.jl#236 · 19 comments · Opened Jul 15, 2021
Update const BENCHMARKTOOLS_VERSION ? Open

bug

Difficulty 1/5 Under an hour Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#233 · 1 comment · Opened Jul 12, 2021
[FR] Benchmark series Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#220 · 4 comments · 3 reactions · Opened Jun 5, 2021
Errors in Sample file Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 50/100

JuliaCI/BenchmarkTools.jl#208 · Opened May 14, 2021
$x should become $(Ref(x))[]? Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#205 · 8 comments · 4 reactions · Opened Apr 14, 2021
Reproducible randomness in setup for BenchmarkGroups Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

JuliaCI/BenchmarkTools.jl#201 · 2 comments · Opened Mar 19, 2021
parallel execution Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 20/100

JuliaCI/BenchmarkTools.jl#186 · 2 comments · Opened Dec 31, 2020
Reusing results from `tune!` Open

question

Difficulty 3/5 1-2 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#182 · 1 comment · Opened Nov 4, 2020
Feature Idea: Custom Benchmarking Metric Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

JuliaCI/BenchmarkTools.jl#176 · 3 comments · 2 reactions · Opened Sep 24, 2020
Pretty-print results from BenchmarkGroup Open

enhancement

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#171 · 3 comments · Opened Jul 8, 2020
Warn against possible code invalidations Open

documentation

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

JuliaCI/BenchmarkTools.jl#167 · 1 comment · Opened May 23, 2020
InexactError when running on 32-bit platform Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 35/100

JuliaCI/BenchmarkTools.jl#166 · 2 comments · 2 reactions · Opened May 9, 2020
is this correct samples with seconds behavior? - better document effect of tuning Open

documentation

Difficulty 3/5 1-2 days Newbie friendliness 38/100

JuliaCI/BenchmarkTools.jl#162 · 4 comments · Opened Apr 14, 2020
Add a simple example of using the teardown phase Open

documentation

Difficulty 2/5 1-3 hours Newbie friendliness 42/100

JuliaCI/BenchmarkTools.jl#160 · 1 comment · 4 reactions · Opened Feb 24, 2020
Clarification questions on design Open

question

Difficulty 5/5 Over a week Newbie friendliness 20/100

JuliaCI/BenchmarkTools.jl#154 · Opened Jan 29, 2020

JuliaCI/BenchmarkTools.jl

Get new issues in your inbox