apple / apple/swift-container-plugin

End to end and integration tests should be run for a controlled matrix of compiler releases

Open
#80 0 comments 0 reactions 0 assignees View on GitHub
area/testing kind/administration
Dominant language
Swift
Stars
390
Forks
26
Avg merge
8m
Merged PRs (30d)
1

Description

The end to end and integration tests use the Swift compiler provided by the GitHub CI runner. The runner image was updated to include Swift 6.1 (https://github.com/actions/runner-images/blob/ubuntu24/20250406.1/images/ubuntu/Ubuntu2404-Readme.md) but this caused the end to end and integration tests to fail because they install the 6.0.2 release of the Static Linux SDK, which is not compatible with Swift 6.1.

#79 fixes this problem in the short term by updating the workflows to a compatible version of the Static Linux SDK, but ideally these tests should not depend on a the particular version of Swift in a runner image, and should also test multiple versions.

These tests do not run in containers because part of the test involves building and running a container image, which is inconvenient when the test itself runs in a container, as used for the matrix unit tests.

Possible solutions:
* run the end to end and integration tests in privileged containers and pass the Docker socket through so images can be built and run
* run the tests in Docker-in-Docker, which can make it easier to set up and tear down such recursive containers.
* Use named runner images with known Swift compilers instead of `ubuntu-latest`.
* Install specific compilers using `swiftly`.

Contributor guide

Open the contributing guide

Research direction

Start by reading the GitHub CI workflows for the end-to-end and integration tests and the changes in #79. Compare the listed approaches—privileged containers, Docker-in-Docker, named runner images, or swiftly-installed compilers—and determine how a controlled compiler matrix can still build and run container images. Done means these tests no longer depend on ubuntu-latest's Swift version and cover multiple compatible compiler releases.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, github-actions, swift
Domain
ci-cd, devops, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.