Support to manually aggregate reports? (e.g. after joining parallel jobs?)
- Dominant language
- Python
- Stars
- 66
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
## Problem
I'd like to aggregate reports across multiple processes, e.g. using `multiprocessing.Pool`.
(In my case, it's a manual worker dispatch based on `mp.Process` - [see here](https://github.com/EricCousineau-TRI/repro/blob/54494a5c5154f19e693e4862fbaa79cddcd78d6f/python/process_util_example/process_util.py#L149) for an old port of the code).
## Motivating Example
In my current code, I do something like this:
```py
def worker(values):
for value in values:
out, timing = do_work(value)
yield out, timing
def main():
values = range(n) # etc.
results = parallel_work(worker, values)
outs, timings = zip(*results)
# Print aggregation of "timings" reports.
```
More concretely, here's the example code (doesn't have all deps, but it communicates the intent):
https://github.com/EricCousineau-TRI/repro/blob/54494a5c5154f19e693e4862fbaa79cddcd78d6f/drake_stuff/multibody_plant_prototypes/generate_poses_sink_clutter.py#L507-L516
## Request
Is there an easy way to aggregate results themselves using public API?
Currently, it looks like aggregation is done internally:
https://github.com/dropbox/stopwatch/blob/94f59aa2134774e06281cf0d02057bb15c1a2f2d/stopwatch.py#L303-L306
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.