actions / actions/actions-runner-controller

Add workflow run times to ARC metrics

Open
#2,359 2 comments 19 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement needs triage
Dominant language
Go
Stars
6.5k
Forks
1.5k
Avg merge
2d 2h
Merged PRs (30d)
27

Description

What would you like added?

In addition to the labels discussed under https://github.com/actions/actions-runner-controller/issues/2176 and implemented in https://github.com/actions/actions-runner-controller/pull/2218 and https://github.com/actions/actions-runner-controller/pull/2225 it would also be good to see workflow related metrics reported by the metrics server. These could be collected from the workflow_run events.

In addition to:

github_workflow_job_run_duration_seconds_bucket
github_workflow_job_run_duration_seconds_count
github_workflow_job_run_duration_seconds_sum

github_workflow_jobs_started_total
github_workflow_jobs_completed_total

the following would also be useful to see:

github_workflow_run_duration_seconds_bucket
github_workflow_run_duration_seconds_count
github_workflow_run_duration_seconds_sum

github_workflows_started_total
github_workflows_completed_total

I might try to open a PR for this unless someone quicker beats me to it.

Why is this needed?

The sum of each job run times of a given workflow isn't equal to the actual time the workflow took to finish because of parallel jobs, queuing of the jobs, etc.

If I wanted to measure how long engineers in our organisation need to wait for CI, workflow run times make more sense for us. If our CI team improved a shared or required workflow by replacing/rewriting a job or an action, or changed anything around the (self-hosted) infrastructure e.g. using larger nodes, changing the autoscaling, etc. we would like to measure what the impact of those changes were, if any.

Also, if we wanted to feed into our business metrics by measuring the time a PR took from start to finish including how much time CI took in the process, workflow run times would be better than individual job run times.

Additional context

In my organisation we're working on our own version of workflow and job metrics, but it has its own issues and adds TOIL to the team which could be discarded if the ARC provided these numbers out of the box.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the metrics server's handling of workflow_run events and the existing workflow-job metrics. Verify that workflow duration histograms and workflow started/completed counters are exposed, then run or add coverage for the resulting metrics if the relevant tests are found.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, go, kubernetes
Domain
observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.