Investigate GitHub Action runners on our infrastructure

Forking discussion from https://github.com/nodejs/build/issues/2205 to keep that issue

@jkleinsc:

@rvagg I think moving over to GitHub actions could be a great way to reduce the maintenance load on the build WG as it is infrastructure you do not have to maintain. The self hosted runners would have to still be maintained, but I'm pretty sure the agent that runs on the self host runners is self updating. The only caveat that might be an issue is that GitHub doesn't recommend using self hosted runners on public repos: help.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners#self-hosted-runner-security-with-public-repositories. I do wonder though if docker containers were used if it would mitigate the security issue.

@mmarchini:

I'm interested in seeing a POC of a runner so we can compare maintenance burden with Jenkins. From a configuration perspective, setting up Actions jobs seems a lot more straightforward than Jenkins.

OTOH, we can't move things permanently from Jenkins to Actions right now as we first need to make sure we can cover the minimum compiler and operating system versions on Actions. Even if we can cover, there are several platforms not supported by Actions (FreeBSD, SmartOS, AIX, PPC, LinuxOne). With that being said, the runner is open sourced, so adding support for these platforms is possible (of course, I don't expect the WG to do it).

Another thing to consider is that our Release workflow relies on Jenkins today, so even if we decide to add runners for the supported platforms, we need to keep the Jenkins jobs to make sure Release still works. Having duplicated machines (one running Jenkins, one running Actions) seems like a bigger maintenance burden though.

Other small issues like lack of custom parametrized runs on Actions also prevents us from moving away from Jenkins (stress tests, for example).

I'm still hopeful for GitHub Actions, but any move in that direction means a bigger change on how we operate and deserves it's own Strategic Initiative. Which is why I'm interested in seeing how a POC will play out, we might find out it's not worth starting any migrations here, or we might find out the maintenance burden is significantly lower, justifying a migration.

Start by reviewing the discussion forked from nodejs/build#2205 and the linked GitHub Actions self-hosted runner security guidance. No files or tests are named; the issue calls for a POC comparing Actions with Jenkins, including supported platforms, release workflows, security, and maintenance burden. Done would require a documented comparison or recommendation.