emscripten-core / emscripten-core/emscripten
Should we make out CI tests immune network failures?
- Dominant language
- C++
- Stars
- 27.6k
- Forks
- 3.6k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 105
Description
For a while today it seem that https://lua.org/ was completely down. This meant that our CI tests were failing becasue we have tests that download and build each of our ports (lua being one of them).
This kind of thing happens from occasionally and its can be very disruptive to our workflow.
One thing we did in the past to mitigate this is that we mirrors all the archive from sourceforge.net into our Google Cloud Storage bucket (which we already rely on in many places). This is because sourceforge.net was having outages every few weeks/months.
To mitigate this further have a couple more options:
1. Move all our ports to use GCS
2. Implement some kind of local caching so that tests don't actually hit the real network.
Any other ideas?
Contributor guide
Research direction
Start by locating the CI tests that download and build each port, then trace how the existing SourceForge archives are mirrored in the Google Cloud Storage bucket. The issue does not choose between moving ports to GCS and adding local caching; done would mean the tests no longer fail when an upstream download site is unavailable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, google-cloud
- Domain
- build-system, ci-cd, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100