emscripten-core / emscripten-core/emscripten

Should we make out CI tests immune network failures?

Open
#27,500 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
27.6k
Forks
3.6k
Avg merge
1d 1h
Merged PRs (30d)
105

Description

For a while today it seem that https://lua.org/ was completely down. This meant that our CI tests were failing becasue we have tests that download and build each of our ports (lua being one of them).

This kind of thing happens from occasionally and its can be very disruptive to our workflow.

One thing we did in the past to mitigate this is that we mirrors all the archive from sourceforge.net into our Google Cloud Storage bucket (which we already rely on in many places). This is because sourceforge.net was having outages every few weeks/months.

To mitigate this further have a couple more options:

1. Move all our ports to use GCS
2. Implement some kind of local caching so that tests don't actually hit the real network.

Any other ideas?

Contributor guide

Open the contributing guide

Research direction

Start by locating the CI tests that download and build each port, then trace how the existing SourceForge archives are mirrored in the Google Cloud Storage bucket. The issue does not choose between moving ports to GCS and adding local caching; done would mean the tests no longer fail when an upstream download site is unavailable.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, google-cloud
Domain
build-system, ci-cd, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.