Use-after-free in epoll muxnote bookkeeping under concurrent `FileHandle.readabilityHandler` teardown (Linux)

Reproducer

Summary

On Linux, rapidly setting up and tearing down FileHandle.readabilityHandler from multiple threads corrupts libdispatch's epoll muxnote state, and the libdispatch manager thread ("DispatchWorker") crashes with SIGSEGV dereferencing a freed/garbage dispatch_muxnote. It reproduces with pure Foundation (no subprocess) — readabilityHandler installs a libdispatch read source on a dup() of the fd, and the concurrent register/teardown churn leaves a dispatch_muxnote freed while it is still reachable (via the _dispatch_sources hash bucket list and/or a live epoll registration). This is not a client-side close-before-cancel misuse: FileHandle monitors a private dup() and closes it only in the source's cancel handler, so the dup guard is correct.

Environment

• Swift 6.5-dev (nightly main), +assertions — Docker image swiftlang/swift:nightly-main-noble
• aarch64-unknown-linux-gnu, Ubuntu 24.04, kernel 6.14.9, running in an aarch64 Linux VM (10 CPUs / 16 GB)

Reproducer

Pure Foundation, no subprocess. Each iteration installs a readabilityHandler (which dups the fd and arms a read source), reads via availableData and clears the handler on EOF, then writes a few bytes and closes the write end so the source fires and tears down. Many of these run concurrently under CPU pressure.

import Foundation
#if canImport(Glibc)
import Glibc
#elseif canImport(Musl)
import Musl
#endif

let durationSeconds = Double(CommandLine.arguments.dropFirst().first ?? "600") ?? 600
let workerThreads = Int(CommandLine.arguments.dropFirst(2).first ?? "24") ?? 24

let deadline = Date().addingTimeInterval(durationSeconds)
let statsLock = NSLock()
var completedCycles = 0

@inline(never)
func stressOneReadabilityHandler() {
  let pipe = Pipe()
  let readEnd = pipe.fileHandleForReading
  let writeEnd = pipe.fileHandleForWriting

  readEnd.readabilityHandler = { handle in
    let data = handle.availableData
    if data.isEmpty {
      handle.readabilityHandler = nil   // EOF: cancel the read source
    }
  }

  try? writeEnd.write(contentsOf: Data([0x41, 0x42, 0x43, 0x0a]))
  try? writeEnd.close()
}

var workers: [Thread] = []
for _ in 0..<workerThreads {
  let worker = Thread {
    while Date() < deadline {
      stressOneReadabilityHandler()
      statsLock.lock(); completedCycles += 1; statsLock.unlock()
      usleep(20)   // let async teardown drain so live fds stay bounded
    }
  }
  worker.stackSize = 4 << 20
  workers.append(worker)
  worker.start()
}

while Date() < deadline {
  sleep(5)
  statsLock.lock(); let cycles = completedCycles; statsLock.unlock()
  FileHandle.standardError.write(Data("cycles=\(cycles)\n".utf8))
}

print("Finished \(completedCycles) cycles without crashing.")

Steps to reproduce

swiftc -O ReadabilityHandlerCrash.swift -o readability-crash
for i in $(seq "$(nproc)"); do yes >/dev/null & done   # saturate CPU
ulimit -c unlimited; ulimit -n 200000
./readability-crash 600 24

Or fully containerized against the nightly image (this is how it was reproduced; works with docker run in place of container run, dropping --platform if the host is already aarch64):

container run --rm \
  --platform linux/arm64 --cpus 10 --memory 16g \
  --volume "$PWD:/repro" \
  swiftlang/swift:nightly-main-noble \
  bash -c '
    ulimit -c unlimited
    ulimit -n 200000
    echo "/repro/core.%e.%p" > /proc/sys/kernel/core_pattern
    export SWIFT_BACKTRACE=enable=no
    cd /repro
    swiftc -O ReadabilityHandlerCrash.swift -o readability-crash
    for i in $(seq "$(nproc)"); do yes >/dev/null & done
    ./readability-crash 1800 24
  '

The process exits with code 139 (SIGSEGV) and writes core.DispatchWorker.<pid> into the mounted directory. Capturing the core requires the container to permit writing /proc/sys/kernel/core_pattern (may need --privileged under some Docker setups, or set core_pattern on the host); the crash reproduces regardless.

It reproduces far more reliably under CPU pressure. In two runs it crashed at ~36.5M cycles (~9 minutes) and ~72.5M cycles (~18 minutes) with 24 worker threads plus nproc CPU burners; budget 15–30 minutes under load. With an idle machine it may not reproduce.

Backtraces

The crash lands on whichever muxnote access loses the race; both sites dereference a non-canonical/garbage pointer on the manager thread.

Registration path (this reproducer), inside _dispatch_muxnote_find's bucket walk:

Thread "DispatchWorker": SIGSEGV, fault address 0x0000fff07ff83ed7
  #0 _dispatch_unote_register_muxed + 220   libdispatch.so
  swiftlang/swift-corelibs-foundation#1 _dispatch_source_invoke        + 480
  swiftlang/swift-corelibs-foundation#2 _dispatch_lane_serial_drain    + 272
  swiftlang/swift-corelibs-foundation#3 _dispatch_mgr_invoke           + 164
  swiftlang/swift-corelibs-foundation#4 _dispatch_mgr_thread           + 132
  swiftlang/swift-corelibs-foundation#5 _dispatch_worker_thread        + 644

+220 is the dmn->dmn_ident load inside LIST_FOREACH(dmn, dmb, dmn_list) in _dispatch_muxnote_find (src/event/event_epoll.c): the walk starts from a valid bucket head and follows dmn_list.le_next into a freed/garbage muxnote.

Delivery path (originally observed in CI while tearing down a subprocess readabilityHandler), inside _dispatch_event_merge_fd inlined into the epoll drain:

Thread "DispatchWorker": SIGSEGV
  #0 _dispatch_event_loop_drain + ~1228     libdispatch.so
  swiftlang/swift-corelibs-foundation#1 _dispatch_mgr_invoke
  swiftlang/swift-corelibs-foundation#2 _dispatch_mgr_thread

Here epoll_wait returned an event whose data.ptr is a muxnote that had already been unlinked from _dispatch_sources and free()d (confirmed via core dump: the muxnote was absent from the live _dispatch_sources[] buckets, and its fields were reused-heap garbage); _dispatch_retain_unote_owner then ~-decodes the reused du_owner_wref to a garbage owner and faults incrementing its refcount.

Analysis

Concurrent read-source registration together with EOF-driven teardown leaves the epoll muxnote lifecycle inconsistent: a dispatch_muxnote is freed while still reachable. On EOF the pipe read end is persistently readable, so the EV_DISPATCH source re-arms and epoll keeps delivering, while the handler concurrently does readabilityHandler = nil (cancel → _dispatch_muxnote_dispose). The registration-path backtrace additionally shows the _dispatch_sources bucket list itself being corrupted (a freed muxnote left linked), so the failure is not limited to stale epoll delivery. The specific in-handler read call is irrelevant — read(upToCount:) reproduces the same crash — the defect is in the read source's registration/teardown, not in how bytes are consumed.

Start with src/event/event_epoll.c, especially _dispatch_muxnote_find, _dispatch_unote_register_muxed, and the epoll drain paths described in the backtraces. Compile the provided ReadabilityHandlerCrash.swift reproducer with Swift 6.5-dev and run it under CPU pressure to observe the failure. Done means the concurrent readabilityHandler setup and teardown no longer crashes or leaves freed muxnotes reachable.