huggingface / huggingface/swift-transformers
Heads-up: release-mode crash when co-linked with vapor/postgres-nio; upstream tracker swiftlang/swift#88794
- Dominant language
- Swift
- Stars
- 1.4k
- Forks
- 209
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 4
Description
# Heads-up: release-mode crash when co-linked with vapor/postgres-nio
Filing as a courtesy / FYI — this looks like a Swift release-mode codegen issue, not a defect in swift-transformers. Upstream tracker: **[swiftlang/swift#88794](https://github.com/swiftlang/swift/issues/88794)**.
## Symptom
A Swift release build that links **both** `huggingface/swift-transformers` (`Tokenizers` product, 1.3.0) and `vapor/postgres-nio` (1.33.0) into the same executable crashes with `freed pointer was not the last allocation` on the **second consecutive `PostgresClient.query(...)` call**. The first query succeeds; the second aborts the process.
Either dependency on its own works fine in release. A `swift build -c debug` build with both deps also works fine.
The trigger is `import Tokenizers` itself — no Tokenizers API has to be called for the crash to fire. That's what points the finger at `Tokenizers`-side codegen / module-init being part of the interaction (alongside NIO's allocator paths).
## Environment
- Swift 6.3.1 (`swiftlang-6.3.1.1.2`) / Xcode 26.4.1 / macOS 26.4.1, Apple M4
- swift-transformers 1.3.0
- postgres-nio 1.33.0
- swift-nio 2.99.0
## Minimal repro
The full minimal Package.swift + `main.swift` (≈30 lines) is in [swiftlang/swift#88794](https://github.com/swiftlang/swift/issues/88794). Two `SELECT N` queries against the bare `PostgresClient` are enough to reproduce, but only when `import Tokenizers` is also in the executable.
## Why I'm posting here
Pure FYI in case:
1. There's a known interaction between Tokenizers' module-init path (Hub/Tokenizers globals, lazy resource lookup, etc.) and Swift's allocator that the swift-transformers maintainers have context on, or
2. you want this on your radar in case other users hit it.
I'm not asking swift-transformers to fix this; the diagnosis points at the Swift toolchain. Feel free to close after triage. If a workaround on the swift-transformers side is feasible (e.g. avoiding whatever module-init path triggers it in release mode), that'd be a nice-to-have, but a Swift compiler patch is the right durable fix.
## Workarounds for users hitting this today
- Build the executable with `swift build -c debug` when both libraries are linked.
- Avoid linking `Tokenizers` and `PostgresNIO` into the same executable target (run tokenization in a separate process, or do tokenization Python-side and ship token IDs to the Swift side).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked upstream tracker swiftlang/swift#88794 and its minimal Package.swift plus main.swift reproduction. Run the release build with both dependencies and the two PostgresClient.query calls; this issue has no requested swift-transformers change, so completion depends on upstream diagnosis or a confirmed workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgres, swift
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100