swiftlang / swiftlang/swift-subprocess
Improve file descriptor disinheritance performance
Nobody has claimed this yet.
- Dominant language
- Swift
- Stars
- 734
- Forks
- 68
- Avg merge
- 9d 2h
- Merged PRs (30d)
- 6
Description
Wrote this up a while back and pasting it here so we can consider applying some of the APIs described here to maximize file descriptor disinheritance performance across various platforms.
--
In POSIX, spawning a new process inherits the parent's file descriptors by default. This is widely considered a security issue, and many libraries (including swift-subprocess) attempt to effectively flip this default.
Unfortunately, preventing subprocesses from inheriting all open file descriptors requires using non-portable API and comes with a significant performance penalty depending on the platform and libc versions. In most cases, swift-subprocess will attempt to use posix_spawn in cases where the platform offers a related posix_spawn API for efficiently closing all open file descriptors. If the platform offers no such API, it will fall back to fork+exec and use a related API for efficiently closing all open file descriptors, and finally fall back to manually computing the largest open file descriptor and manually issuing a potentially very large number of close() syscalls.
The following table indicates the performance and techniques used based on the spawn strategy used. The posix_spawn column indicates the specific O(1) posix_spawn API used to close all file descriptors. The pre-fork column indicates the file descriptor closing strategy used when posix_spawn can't be used. - in both columns indicates that O(n) manual closing will be used.
| Platform | Version | posix_spawn | pre-fork |
|---|---|---|---|
| macOS | 10.7+ | posix_spawnattr_setflags + POSIX_SPAWN_CLOEXEC_DEFAULT | posix_spawnattr_setflags + POSIX_SPAWN_CLOEXEC_DEFAULT |
| Linux | kernel 5.9+ with Glibc 2.34+ | posix_spawn_file_actions_addclosefrom_np | close_range |
| ^ | kernel 5.9+ with older Glibc | - | close_range |
| ^ | kernel 5.9+ with Musl | - | close_range |
| ^ | kernel < 5.9 | - | - |
| Android | kernel 5.11+ | posix_spawnattr_setflags + POSIX_SPAWN_CLOEXEC_DEFAULT | close_range |
| ^ | kernel < 5.11 | posix_spawnattr_setflags + POSIX_SPAWN_CLOEXEC_DEFAULT * | - |
| FreeBSD | 13.1+ | posix_spawn_file_actions_addclosefrom_np | close_range / closefrom |
| OpenBSD | 3.5+ | - | closefrom |
* POSIX_SPAWN_CLOEXEC_DEFAULT requires Android 13 (API level 33) and provides no good programmatic way to know if it's supported other than checking the version at runtime
* Android with a kernel older than Linux 5.11 falls back to an O(n) solution when using posix_spawn because close_range with CLOSE_RANGE_CLOEXEC is not available.
* Thanks to POSIX_SPAWN_SETEXEC, we never need to use raw exec on macOS even in the pre-fork case.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files or tests. Review swift-subprocess's existing posix_spawn and fork+exec paths against the platform and API table, then verify that supported platforms use the most efficient available file-descriptor disinheritance strategy while preserving fallbacks for older systems.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- operating-systems, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100