Marvin v3: Tracking a PR's lifecycle

Open
#371,069 20 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
20/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
github-actions
Domain
ci-cd, tooling

Research direction

No files or tests are named. Start by evaluating the proposed GitHub Actions or App approach against the listed PR states and label transitions, including unresolved review comments and opt-out behavior. Done means the lifecycle and label set are agreed and an implementation path is defined.

Written by the indexing model from the issue text.

Description

0.kind: enhancement 2.status: backlog 2.status: stale 6.topic: continuous integration 6.topic: developer experience

Issue description

In 2022-04-16 at 12:48:13 UTC, Marvin was declared dead. For those unaware, Marvin was an experimental GitHub app that tagged Nixpkgs PRs and was used by maintainers and committers to triage PRs. New PRs would be tagged with awaiting_reviewer, then awaiting_changes after a review has been posted, and finally awaiting_merger after an approval.

Last year, the old Marvin labels began to be manually used in PRs. This somewhat polluted Marvin's historical log, and isn't an ideal solution, considering how the labels are undocumented and using the default #FFFFFF scheme, which is usually reserved for 5.scope labels. This clearly shows the need for a new version of Marvin: a PR lifecycle tracker that clearly displays which PRs need reviewers, have been awaiting reviewers, or are waiting on the PR author for changes.

Thanks to the power of hindsight, we can avoid Marvin v2's biggest issue: the spam. Marvin would request reviews, ping maintainers, and leave superfluous comments in the PRs it managed. I propose a simple labeler that does nothing but automate the work that @FliegendeWurst has been doing, which will allow us to filter PRs more effectively. This also allows us to design a much more simple solution with far less features.

Proposed solution

Leverage GitHub Actions or an App to add the following labels to PRs:

4.stage: awaiting reviewers, for when a PR has no reviewers after nix-owners runs.
4.stage: awaiting review, for when a PR has review requests, but no published reviews yet.
4.stage: awaiting changes, for when there is any published review with unresolved comments. (TODO: can we track resolved/unresolved review comments?)
4.stage: awaiting merger, for approved PRs which have not been interacted with by a committer.
4.stage: awaiting merge, for approved PRs which have been reviewed by a committer or have requested the review from a committer. Both of these will mimic the 12.approvals series of labels.

Extra: move unresolved_build_failure to 2.status or 9.needs. I'm not sure if that's Marvin-related, but it sure looks like it, and it could do with a touch-up.

This list is not final and is always up for discussion and improvement. It's a more-or-less 1:1 replacement for Marvin, but it could be argued that awaiting merge(r) doesn't need to exist now that 12.approvals is a thing. I'd also be up for adding more labels for more PR states, such as when the changes were addressed but no approval was given yet.

There should also be a way to opt-out from labeling, and another label from when the usual lifecycle is skipped. For instance, a PR like this could be given a 4.stage: reviews skipped label, which would allow us to track trivial, tested PRs from package maintainers, and perhaps slightly improve breakage tracking if someone has been reckless.


Add a 👍 reaction to issues you find important.

Dominant language
Nix
Stars
26.2k
Forks
20.1k
Avg merge
9h 57m
Merged PRs (30d)
517

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NixOS/nixpkgs

All issues in NixOS/nixpkgs

Similar issues

More DevOps issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.