dask / dask/dask-benchmarks

Documentation

Open
#1 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
18
Forks
19
PR merge metrics
No merged PRs in 30d

Description

This is a sketch for some sections of documentation that should go in the README.

## What to test?

Ideally, benchmarks measure how long *our project* (dask, distributed) spends doing something, not the underlying libraries they're built on. We want to limit the variance across runs to just code we control.

For example, I suspect `(self.data.a > 0).compute()` is not a great benchmark. My *guess* (without having profiled) is that the `.compute` part takes the majority of the time, most of which would be in pandas / NumPy. (I need to profile all these. I'm reading through dask now to find places where dask is doing a lot of work.)

## Benchmarking new Code

If you're writing an optimization, say, you can benchmark it by

- writing a benchmark that exercises your optimization and placing it in `benchmarks/`
- setting the `repo` field in `asv.conf.json` to the path of your dask / distributed repository on your local file system
- running `asv continuous -f 1.1 upstream/master HEAD` (optionally with a regex `-b ` to filter to just your benchmark.

## Naming Conventions

## Directory Structure

This repository contains benchmarks for several dask related projects.
Each project needs it's own benchmark directory because `asv` is built around
one configuration file (`asv.conf.json`) and benchmark suite per repository.

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing README, the benchmarks/ directory, and asv.conf.json to understand the repository structure and benchmark commands. Turn the sketch into README sections covering what to benchmark, how to benchmark new code, naming conventions, and directory structure; use the documented asv continuous command to verify the instructions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, performance
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.