guardian / guardian/prout

Prout ascribes Testing-in-Production results to a PR before the PR is even seen in Prod

Open
#43 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
147
Forks
15
Avg merge
3d 7h
Merged PRs (30d)
2

Description

In membership frontend, 2 PRs were merged within a few minutes of each other:

1. [PR 1436](https://github.com/guardian/membership-frontend/pull/1436)
2. [PR 1441](https://github.com/guardian/membership-frontend/pull/1441)

Although PR 1436 went out first, _all_ the Testing-In-Production results were posted by Prout onto PR 1441. Though it can be hard to accurately ascribe a test to PR, what went **particularity** wrong here was that the TiP results [were ascribed](https://github.com/guardian/membership-frontend/pull/1441#issuecomment-273739485) to PR 1441 before it was even _seen_ in Production:

![image](https://cloud.githubusercontent.com/assets/52038/22104248/3a6f441a-de36-11e6-9e4e-6b71e06b9abc.png)

Contributor guide

Open the contributing guide

Research direction

Start by comparing the deployment and production-visibility timing for PR 1436 and PR 1441, then inspect the TiP result attribution path. Reproduce the case where results are attached to PR 1441 before it is seen in production; done means the results are attributed to the appropriate PR only after the relevant production evidence is available.

Written by the indexing model from the issue text.

Assessment

Tech stack
github, scala
Domain
devops, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.