swiftlang / swiftlang/swift-subprocess

Improve file descriptor disinheritance performance

Open
#172 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Swift
Stars
734
Forks
68
Avg merge
9d 2h
Merged PRs (30d)
6

Description

Wrote this up a while back and pasting it here so we can consider applying some of the APIs described here to maximize file descriptor disinheritance performance across various platforms.

--

In POSIX, spawning a new process inherits the parent's file descriptors by default. This is widely considered a security issue, and many libraries (including swift-subprocess) attempt to effectively flip this default.

Unfortunately, preventing subprocesses from inheriting all open file descriptors requires using non-portable API and comes with a significant performance penalty depending on the platform and libc versions. In most cases, swift-subprocess will attempt to use posix_spawn in cases where the platform offers a related posix_spawn API for efficiently closing all open file descriptors. If the platform offers no such API, it will fall back to fork+exec and use a related API for efficiently closing all open file descriptors, and finally fall back to manually computing the largest open file descriptor and manually issuing a potentially very large number of close() syscalls.

The following table indicates the performance and techniques used based on the spawn strategy used. The posix_spawn column indicates the specific O(1) posix_spawn API used to close all file descriptors. The pre-fork column indicates the file descriptor closing strategy used when posix_spawn can't be used. - in both columns indicates that O(n) manual closing will be used.

Platform Version posix_spawn pre-fork
macOS 10.7+ posix_spawnattr_setflags + POSIX_SPAWN_CLOEXEC_DEFAULT posix_spawnattr_setflags + POSIX_SPAWN_CLOEXEC_DEFAULT
Linux kernel 5.9+ with Glibc 2.34+ posix_spawn_file_actions_addclosefrom_np close_range
^ kernel 5.9+ with older Glibc - close_range
^ kernel 5.9+ with Musl - close_range
^ kernel < 5.9 - -
Android kernel 5.11+ posix_spawnattr_setflags + POSIX_SPAWN_CLOEXEC_DEFAULT close_range
^ kernel < 5.11 posix_spawnattr_setflags + POSIX_SPAWN_CLOEXEC_DEFAULT * -
FreeBSD 13.1+ posix_spawn_file_actions_addclosefrom_np close_range / closefrom
OpenBSD 3.5+ - closefrom

* POSIX_SPAWN_CLOEXEC_DEFAULT requires Android 13 (API level 33) and provides no good programmatic way to know if it's supported other than checking the version at runtime
* Android with a kernel older than Linux 5.11 falls back to an O(n) solution when using posix_spawn because close_range with CLOSE_RANGE_CLOEXEC is not available.
* Thanks to POSIX_SPAWN_SETEXEC, we never need to use raw exec on macOS even in the pre-fork case.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files or tests. Review swift-subprocess's existing posix_spawn and fork+exec paths against the platform and API table, then verify that supported platforms use the most efficient available file-descriptor disinheritance strategy while preserving fallbacks for older systems.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
operating-systems, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.