Benchmark for graph construction
- Dominant language
- Python
- Stars
- 18
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Description
While exploring scheduler improvements it was recently discovered that removing redundant string concatenations has a measurable performance increase. Tom noted that there currently were no isolated benchmarks which [tested graph construction](https://github.com/dask/dask/pull/6137#issuecomment-624865967)
We could do something similar to what was suggested in https://github.com/dask/dask/pull/6137
```python
In [2]: ddf_d = timeseries(start='2000-01-01', end='2002-01-01', partition_freq='1d')
In [3]: %timeit shuffle(ddf_d, "id", shuffle="tasks")
67.2 ms ± 2.54 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
```
Contributor guide
Research direction
The issue names no benchmark file; start by reviewing the repository's existing ASV benchmarks and reproduce the supplied Python example for Dask graph construction. Done means an isolated benchmark measures graph construction, including the shuffle example, so performance changes such as removing redundant string concatenations can be compared.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100