JuliaCI / JuliaCI/BenchmarkTools.jl

LinuxPerf, @profile, and other experiments

Open
#377 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
684
Forks
107
PR merge metrics
No merged PRs in 30d

Description

This issue is to document various PRs surrounding Linuxperf and other extensible benchmarking in Benchmarktools. I've seen many great approaches, with various differences in semantics and interfaces. It seems that https://github.com/JuliaCI/BenchmarkTools.jl/pull/375 profiles each eval loop (toggling on and off with a boolean), https://github.com/JuliaCI/BenchmarkTools.jl/pull/347 is a generalized version of the same (unclear whether this can generalize to more than one extension at a time, such as profiling and perfing), and https://github.com/JuliaCI/BenchmarkTools.jl/pull/325 only perfs a single execution.I recognize that different experiments require different setups. A sampling profiler requires a warmup and a minimum runtime, but probably doesn't need fancy tuning. A wall-clock time benchmark requires a warmup and a fancy eval loop where the evaluations are tuned, and maybe a gc scrub. What does LinuxPerf actually need? Are there any other experiments we also want to run (other than linuxperf?). Do we need to use metaprogramming to inline the LinuxPerf calls, or are function calls wrapping the samplefunc sufficient here?
@vchuravy @DilumAluthge @topolarity @Zentrik

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing PRs #375, #347, and #325, focusing on how each handles LinuxPerf, profiling, evaluation loops, and extensions. Document the required setup for each experiment, whether multiple extensions can run together, and whether function calls or metaprogramming are appropriate; done means the approaches and open decisions are clearly recorded.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.