dotnet / dotnet/arcade

Helix Job Monitor - Job cancellation fails

Open
#17,502 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
729
Forks
397
Avg merge
3d 15m
Merged PRs (30d)
149

Description

When the Helix Job Monitor is timing out, it fails to cancel jobs:

```
##[warning]Failed to cancel Helix job Windows_NT Build_Release - windows.11.amd64.client.open.svc (019e4529-01a4-4034-8c9a-0434bfa5999e).
     Newtonsoft.Json.JsonReaderException: Error parsing Infinity value. Path '', line 1, position 2.
        at Microsoft.DotNet.Helix.JobMonitor.HelixService.RetryAsync[T](Func`1 action, CancellationToken cancellationToken) in /_/src/Microsoft.DotNet.Helix/JobMonitor/Services/HelixService.cs:line 461
        at Microsoft.DotNet.Helix.JobMonitor.HelixService.CancelJobAsync(String jobName, CancellationToken cancellationToken) in /_/src/Microsoft.DotNet.Helix/JobMonitor/Services/HelixService.cs:line 206
        at Microsoft.DotNet.Helix.JobMonitor.JobMonitorRunner.<>c__DisplayClass24_0.<b__2>d.MoveNext() in /_/src/Microsoft.DotNet.Helix/JobMonitor/JobMonitorRunner.cs:line 670
```

[https://dev.azure.com/dnceng-public/public/_build/results?buildId=1585898&view=logs&j=d0ce146c-3e88…](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1585898&view=logs&j=d0ce146c-3e88-5c77-cddf-9b2bfcb78d0a&s=859b8d9a-8fd6-5a5c-6f5e-f84f1990894e&t=79dc1f37-4627-57e5-8cbd-d046b98e6aba&l=119)

Contributor guide

Open the contributing guide

Research direction

Start in src/Microsoft.DotNet.Helix/JobMonitor/Services/HelixService.cs around RetryAsync (line 461) and CancelJobAsync (line 206), then trace the cancellation call from JobMonitorRunner.cs around line 670. Reproduce or cover the Infinity JSON parsing failure and verify that timed-out Helix jobs cancel without this exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.