rapidsai / rapidsai/deployment

Make documentation testing more scriptable

Open
#549 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
15
Forks
41
Avg merge
1d 8h
Merged PRs (30d)
6

Description

Every release we do two kinds of testing:

  • Documentation testing - Are the instructions in our documentation followable? Or has the world shifted around us requiring us to update someting?
  • Technical testing - Are RAPIDS packages installable and runnable in every environment? Has something changed in the package or the cloud environment which has broken compatibility?

Currently we do both of these manually every release. Documentation testing is hard to automate because it involved consuming instructions written for humans and ensuring humans can still follow them as written.

The technical testing could be more automated. Creating infrastructure, running an install script and then checking functionality could be written as a test.

We should update our documentation pages to format them in a way where scripting would be easier.

One idea is to copy the structure of the Azure VM page where we have tabs to choose between GUI instructions and CLI instructions. This way the CLI instructions can be extracted via a script and run automatically.

One challenge to be aware of is that the potential failure mode of these scripts could result in cloud resources being left hanging and costing money if not closely monitored. We should aim to manually launch any automations at the beginning of our manual testing process and then have a manual step at the end to check and verify that all created resources have been cleaned up.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked Azure VM documentation page and compare its GUI and CLI instruction structure with other deployment documentation pages. Define what documentation can be extracted for automated technical testing, while preserving manual launch and final cloud-resource cleanup checks; done means the documentation supports this scriptable workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
cloud, documentation, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.