JuliaCI / JuliaCI/BenchmarkTools.jl
LinuxPerf, @profile, and other experiments
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 684
- Forks
- 107
- PR merge metrics
- No merged PRs in 30d
Description
This issue is to document various PRs surrounding Linuxperf and other extensible benchmarking in Benchmarktools. I've seen many great approaches, with various differences in semantics and interfaces. It seems that https://github.com/JuliaCI/BenchmarkTools.jl/pull/375 profiles each eval loop (toggling on and off with a boolean), https://github.com/JuliaCI/BenchmarkTools.jl/pull/347 is a generalized version of the same (unclear whether this can generalize to more than one extension at a time, such as profiling and perfing), and https://github.com/JuliaCI/BenchmarkTools.jl/pull/325 only perfs a single execution.I recognize that different experiments require different setups. A sampling profiler requires a warmup and a minimum runtime, but probably doesn't need fancy tuning. A wall-clock time benchmark requires a warmup and a fancy eval loop where the evaluations are tuned, and maybe a gc scrub. What does LinuxPerf actually need? Are there any other experiments we also want to run (other than linuxperf?). Do we need to use metaprogramming to inline the LinuxPerf calls, or are function calls wrapping the samplefunc sufficient here?
@vchuravy @DilumAluthge @topolarity @Zentrik
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing PRs #375, #347, and #325, focusing on how each handles LinuxPerf, profiling, evaluation loops, and extensions. Document the required setup for each experiment, whether multiple extensions can run together, and whether function calls or metaprogramming are appropriate; done means the approaches and open decisions are clearly recorded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- performance
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100