clearlydefined / clearlydefined/operations

approach to evaluate dev deploys fitness to advance to production

Open
#90 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
4
Forks
15
PR merge metrics
No merged PRs in 30d

Description

### Description

Decisions to advance dev deploys to production are evaluated by [current integration tests](https://github.com/clearlydefined/operations/blob/main/tools/integration/README.md) and ad hoc testing on the dev deploys. This issue proposes a systematic way to evaluate the fitness to advance to production.

### Proposal

* randomly select 50 coordinates per package manager impacted by the changes (_from last 12 months only_)
* get current production summary definition for each
* use dev deploys to force generate harvest results and force generate a summary definition
* compare production and dev summary definitions for diffs
* evaluate diffs to determine if they block advancement to production

Factors blocking advancement
* diff is a significant change requiring further evaluation to determine acceptability
* dev summary definition is known to be wrong

### Related Work

* [existing integration tests using fixtures](https://github.com/clearlydefined/operations/blob/main/tools/integration/README.md)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.