bencheeorg / bencheeorg/benchee
Custom collectors
- Dominant language
- Elixir
- Stars
- 1.5k
- Forks
- 70
- PR merge metrics
- No merged PRs in 30d
Description
edit by @PragTob: This is the original issue, we arrived at the solution though that custom collectors would solve this: https://github.com/bencheeorg/benchee/issues/444#issuecomment-2573561240
Hello,
In order to not pollute the computation of something with the process dictionary being already fed with memoized results, I have a function that I would benchmark which boils down to basically this:
```elixir
fn input ->
parent = self()
spawn_link(fn -> send(parent, acutal_computation(input)) end)
receive(do: (msg -> msg))
end
```
Problem is that the "entrypoint" that I can give to benchee must be the external function, to keep the isolation.
Is there a way to provide benchee with a special scenario, something like that:
```elixir
fn input ->
parent = self()
spawn_link(fn ->
_result = acutal_computation(input) # not used for benchmark
measurements = Benchee.some_function_to_capture_measurements()
send(parent, measurements)
end)
# return the measurements to benchee
receive(do: (msg -> msg))
end
```
I want to do that first because the measurements should be taken deeper in the call stack, here it's just for the example. And secondly because for memory consumption ~I don't know if benchee looks at the whole system or just the current process (I guess it does the segond given the code I have read)~. So If the function spawn the memory will be off.
Edit:
> Like memory measurements, this only tracks reductions directly in the function given to benchee, not processes spawned by it or other processes it uses.
Thank you
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked issue comment and the custom collectors approach described in the issue, then inspect Benchee's existing measurement and benchmark entry points. Define how measurements captured inside spawned processes are returned to the benchmark and how completion is verified for memory and reduction metrics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100