box / box/ClusterRunner

Timing data should sometimes be saved for failed builds

Open
#244 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
184
Forks
41
PR merge metrics
No merged PRs in 30d

Description

Currently we only update saved timing data for a job when a build is successful (no failed atoms). This makes sense in general -- failed builds might have unrepresentative timing data.

I think there are situations though when we might want to update timing data even when atoms fail. The situations I was thinking of were
- when no timing data currently exists. When a job config is first created, no timing data exists and it's common for users to still be working out the kinks in their config, so there may be many initial builds that all fail. It would be better if clusterrunner would save timing data even for failed builds if it doesn't have data for a successful build.
- when a build has not passed in a long time. We've seen in practice that old timing data can actually have a worse affect on build times than having no timing data at all. If there is something weird about a job causing builds to fail and nobody notices, this can cause build times to slowly increase over time as the timing data gets more and more outdated.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.