bencheeorg / bencheeorg/benchee

More generic profiler support (eflambe in particular)

Open
#440 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Elixir
Stars
1.5k
Forks
70
PR merge metrics
No merged PRs in 30d

Description

I saw a cool talk at Code BEAM EU about [performance optimizations and the 1 billion row challenge](https://codebeameurope.com/talks/the-one-billion-row-challenge-in-elixir-from-12-minutes-to-25-seconds/) by @rajrajhans and he used [eflambe](https://github.com/Stratus3D/eflambe) for profiling.

So the question was... why not add support for more profilers? Right now we only support built-in profilers.

In true benchee mannerism though I'd want it to be more generic:
* no hard dependency on eflambe, most likely not even an optional one - point to a plugin
* since it's a plugin, we'd need to design a behavior (just like for formatters) to conform with for "profiling plugins"

likely similiar to the interface of the built-in profilers which we already support: https://hexdocs.pm/mix/Mix.Tasks.Profile.Tprof.html

I'd expect that interface to be quite simple, but I'm not looking deeply into it just now :) Here is where/how we currently use them: https://github.com/bencheeorg/benchee/blob/main/lib/benchee/profile.ex#L109

https://github.com/bencheeorg/benchee/blob/main/lib/benchee/benchmark/collect/profile.ex

It boils down to:

```elixir
return_value = profiler_module.profile(function, profiler_opts)
```

With that we could also remove our reliance on the mix tasks (I'm unsure how much benefit they give us over calling the profilers themselves).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading lib/benchee/profile.ex around line 109 and lib/benchee/benchmark/collect/profile.ex, then compare the existing profiler interface with Mix.Tasks.Profile.Tprof. Define the profiling-plugin behavior and its integration points. Done means profilers can be supplied through a generic plugin interface without a hard eflambe dependency or reliance on Mix tasks.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
performance, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.