holoviz / holoviz/datashader

GPU testing of PRs

Open
#1,302 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
3.6k
Forks
376
Avg merge
4h 32m
Merged PRs (30d)
1

Description

This is to document the GPU testing I usually perform for Datashader PRs. Currently there is no automated GPU testing in the GitHub Actions-based CI, so this needs to be performed manually.

Test machine can be local or remote, and needs to have a Nvidia graphics card. Recommended Operating System is Ubuntu, other Linux distros will also work and even Windows via WSL2 but they may be harder to set up. Driver needs to be installed for the graphics card, and the `nvidia-utils` package is useful as it provides the `nvidia-smi` command-line tool.

Run `nvidia-smi` to determine the driver and CUDA version. My results are:
```
Driver Version: 470.199.02 CUDA Version: 11.4
```

You need an appropriate `conda`/`mamba` environment containing `cuDF`. This is most easily obtained by installing RAPIDS following the installation guide at https://docs.rapids.ai/install. Opt for the stable release with the same version of CUDA that you are using, and your preferred choice of Python version (generally the later the better). There is other information on the RAPIDS install page to help if you experience problems.

Install Datashader using `pip`, e.g. from the root directory of your local copy of the datashader github repo:
```
pip install -ve .[tests]
```

GPU tests are not run by default but need an environment variable defined. To run the whole test suite on both CPU and GPU use
```
DATASHADER_TEST_GPU=1 pytest -v datashader/tests
```

Unless you are dealing with a GPU-specific issue, it is recommended for PRs to get everything working and tests written and passing for the CPU before adding GPU support.

There is a PR in progress to add GPU tests on a self-hosted runner: #1301

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue body contains the manual GPU-testing procedure; start by reviewing the repository documentation structure and PR #1301. Run `nvidia-smi` to verify the documented environment, then check that the RAPIDS setup, `pip install -ve .[tests]`, and `DATASHADER_TEST_GPU=1 pytest -v datashader/tests` steps are complete and accurate.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, linux, python
Domain
ci-cd, documentation, testing-qa
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.