TASEmulators / TASEmulators/BizHawk

We're burning through the GitLab CI execution time quota

Open
#3,426 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Meta
Dominant language
C#
Stars
2.8k
Forks
468
PR merge metrics
No merged PRs in 30d

Description

2022-10: We're 70% through the month with 73% used. We're out.
2022-11: Ended with 98%. I feel obliged to thank the other devs for not committing, which... yeah that's not good.
2022-12: Used 96% in 9 days thanks to release rush. I decided to fork out the AU$15 for more CI minutes before we ran out. But then GitLab decided to enforce Pull Mirroring being a paid feature (the docs had been claiming it was already), so we'll need to push manually or set a cron job to do the same. I can't make any uptime guarantees, someone else will have to do it. Used a second GitLab project as a cron job :P It's only hourly though.
2023-01: forgot about the cronjob and turns out "only hourly" was actually way too frequently... oops
2023-02: fixed cronjob, usage for this month only hit 50%
2023-06: The more powerful runners they added last month seem to have helped, we're way down now (some of that would be due to the lack of commits, but the cronjob repo also used fewer minutes).
Screenshot_20230701_154052


live view
Screenshot_20221022_135810
Screenshot_20221022_135848
(This quota was a recent change to the $0 tier.)

Strategy A:
Apply for the GitLab for Open Source program. Its members get the cap raised 125x to 50000 min. This solution isn't cut-and-dried because of license compliance.

Strategy B:
Use less.

  • We're already compiling with all CPU cores (though it seems that still doesn't work on Linux, I did a comparison in CI just now and it was within a few seconds).
  • In fixing the sporadic job failures, I reduced the Git execution time on every job, but that just means there's no room to go lower.
  • The number of jobs run for each new commit is already reduced, extra jobs being run only once a week approx. fortnightly, on the 8th and 23rd of each month.
  • More than half of the package_devbuild_{windows,linux} jobs' execution time was spent on grabbing the Nix Docker image. I've replaced it with a smaller non-Nix Alpine image.
  • Revisit #2180? The thought is that even if it's the same number of source files, even if MSBuild can't do multiple projects in parallel, several smaller projects will still be faster simply because it doesn't need to check for as many cross-file references.
  • Suggestions welcome! Please?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the GitLab CI configuration and the package_devbuild_{windows,linux} jobs, focusing on the Nix image replacement and current job schedule. Read the discussion and proposal in #2180 before considering project-build changes. Done means reducing CI execution minutes without removing required checks or causing job failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, docker, gitlab
Domain
build-system, ci-cd, devops
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.