apple / apple/container

[Bug]: CI build canceled, seeming problem in TestCLIBuildBase.

Open
#1,250 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
49.9k
Forks
1.8k
Avg merge
1d 20h
Merged PRs (30d)
22

Description

### I have done the following

- [x] I have searched the existing issues
- [x] If possible, I've reproduced the issue using the 'main' branch of this project

### Steps to reproduce

This happens intermittently in CI. The `TestCLIRunInitImage` suite runs to completion, and then no additional output appears in the logs (see below), and eventually the action is canceled at 1 hour.

All that's needed to reproduce is to run a PRB, merge, or release build on GH.

### Current behavior

The TestCLIRunInitImage suite starts at 2026-02-22 20:20:18.512 and appears to run until 2026-02-22T20:20:21.562Z based on a log search:

```console
% cat ~/Downloads/logs/* | sort | grep initimage | tail -5
2026-02-22T20:20:21.561Z [debug] com.apple.container APIServer ["id": testrunwithexplicitdefaultinitimage, "acquirer": handleContainerExit(id:code:)]: releasing lock
2026-02-22T20:20:21.562Z [debug] com.apple.container APIServer ["acquirer": handleContainerExit(id:code:), "id": testrunwithexplicitdefaultinitimage]: holding lock
2026-02-22T20:20:21.562Z [debug] com.apple.container APIServer ["acquirer": handleContainerExit(id:code:), "id": testrunwithexplicitdefaultinitimage]: releasing lock
2026-02-22T20:20:21.562Z [debug] com.apple.container APIServer ["id": testrunwithexplicitdefaultinitimage, "func": stop(id:options:)]: ContainersService: exit
2026-02-22T20:20:21.562Z [info] com.apple.container APIServer ["id": testrunwithexplicitdefaultinitimage, "rc": ExitStatus(exitCode: 137, exitedAt: 2026-02-22 20:20:21 +0000)]: handling container exit
```

The tests after TestCLIRunInitImage are TestCLIBuildBase. These tests appear to run until the last captured logs at 2026-02-22T20:22:47.300Z:

```console
% cat ~/Downloads/logs/* | sort | tail -10
2026-02-22T20:22:47.281Z [debug] com.apple.container APIServer ["id": buildkit, "processId": 40a77656-d4d4-42ab-aae7-90db60d118c8, "acquirer": startProcess(id:processID:)]: acquiring lock
2026-02-22T20:22:47.281Z [debug] com.apple.container APIServer ["id": buildkit, "processId": 40a77656-d4d4-42ab-aae7-90db60d118c8, "acquirer": startProcess(id:processID:)]: holding lock
2026-02-22T20:22:47.281Z [debug] com.apple.container APIServer ["id": buildkit, "processId": 40a77656-d4d4-42ab-aae7-90db60d118c8, "func": wait(id:processID:)]: ContainersService: enter
2026-02-22T20:22:47.281Z [debug] com.apple.container RuntimeLinuxHelper ["uuid": buildkit, "func": wait(_:)]: enter
2026-02-22T20:22:47.282Z [debug] com.apple.container RuntimeLinuxHelper ["uuid": buildkit, "func": startProcess(_:)]: enter
2026-02-22T20:22:47.298Z [debug] com.apple.container RuntimeLinuxHelper ["func": startProcess(_:), "uuid": buildkit]: exit
2026-02-22T20:22:47.298Z [error] com.apple.container RuntimeLinuxHelper ["error": internalError: "failed to start process" (cause: "internal error (13): startProcess: failed to start process: internalError: "vmexec error: internalError: "read ack pipe"""), "route": com.apple.container.sandbox/start, "uuid": buildkit]: route handler threw an error
2026-02-22T20:22:47.299Z [debug] com.apple.container APIServer ["func": startProcess(id:processID:), "id": buildkit, "processId": 40a77656-d4d4-42ab-aae7-90db60d118c8]: ContainersService: exit
2026-02-22T20:22:47.299Z [debug] com.apple.container APIServer ["id": buildkit, "processId": 40a77656-d4d4-42ab-aae7-90db60d118c8, "acquirer": startProcess(id:processID:)]: releasing lock
2026-02-22T20:22:47.300Z [error] com.apple.container APIServer ["error": internalError: "failed to start process 40a77656-d4d4-42ab-aae7-90db60d118c8 in container buildkit" (cause: "internalError: "failed to start process (cause: "internal error (13): startProcess: failed to start process: internalError: "vmexec error: internalError: "read ack pipe""")""), "route": containerStartProcess]: route handler threw an error
```

### Expected behavior

Tests should not hang.

### Environment

```markdown
- OS: 26.2
- Xcode: Apple Swift version 6.2.3 (swiftlang-6.2.3.3.21 clang-1700.6.3.2)
- Container: main
```

### Relevant log output

```shell
see above
```

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

Contributor guide

Open the contributing guide

Research direction

Reproduce a PR, merge, or release build on GitHub and inspect the logs around TestCLIRunInitImage and TestCLIBuildBase. Start with the failed buildkit process and the “vmexec error: read ack pipe” output, then trace why the CI run stops producing logs. Done means the affected tests complete without hanging or causing the action to be canceled.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, swift
Domain
ci-cd, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.