cupy / cupy/cupy

Performance monitoring test

Open
#742 2 comments 0 reactions 0 assignees View on GitHub
cat:test prio:high
Dominant language
Python
Stars
12.3k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
45

Description

Performance is an important strength of CuPy.
For better quality assurance, it is nice to have a way to "fixed-point observation" of performance.
By having a baseline set of benchmarks, we can:

* Correctly evaluate performance improvement for each pull-requests
* Detect unexpected performance regression

NumPy has a set of benchmark codes to run on [Airspeed Velocity](https://asv.readthedocs.io/en/latest/).

https://github.com/numpy/numpy/tree/master/benchmarks

I haven't tested them yet, but NumPy benchmark code seems so simple that we can borrow them and then do `s/np/cp/`. It may be a good start point.

https://github.com/numpy/numpy/tree/master/benchmarks/benchmarks

We may however want some additional metrics like:

* memory allocation performance
* comparison with NumPy

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the NumPy benchmark examples linked in benchmarks and benchmarks/benchmarks, along with the Airspeed Velocity documentation. Determine how CuPy benchmarks should establish fixed performance baselines and detect regressions. Done should include a documented benchmark setup that evaluates pull-request performance, with the proposed memory-allocation and NumPy-comparison metrics addressed or scoped.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
performance, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.