microsoft / microsoft/go

Give builders two timeouts: one for expected time, another to see if it would have succeeded if we have given it a higher timeout

Open
#568 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Test
Dominant language
Go
Stars
431
Forks
44
Avg merge
21h 18m
Merged PRs (30d)
30

Description

This is a strategy I've seen used for .NET tests:

  1. Fail the build after an expected maximum time it should take to run the tests on the hardware. Keeping this relatively tight prevents PRs from unknowingly massively regressing the time.
  2. Keep the build running until a significantly longer timeout expires. The build still fails, but this way, we get to see if the build would succeed if we were to extend the timeout.
    • This is very useful when we do expect a performance regression, or when we don't have time to fix it yet. We can see how much longer we need to extend the timeout for, and whether there's a real hang.

The long tests are long. I just had a build where one longtest succeeded at 53 minutes and another failed due to a 1hr timeout--this strategy would have been useful.

This can be done by giving the AzDO job the long timeout, and make eng/run.ps1 run-builder keep track of the shorter timeout.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading eng/run.ps1, especially the run-builder path, and inspect how the AzDO job currently applies its timeout. Define the shorter expected-runtime failure separately from the longer build-running timeout. Done means a build reports an early timeout while continuing until the longer limit, distinguishing a slow success from a hang.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, powershell
Domain
build-system, ci-cd, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.