Go dynamic module deadlocks integration-test server teardown on macOS
- Dominant language
- C++
- Stars
- 28.9k
- Forks
- 5.6k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 430
Description
**Corrected from the original report.** I first described this as the request hanging. It isn't: the request succeeds and the test body passes. The hang is in test *teardown*, and it looks macOS-specific.
With a Go dynamic module loaded, `~IntegrationTestServer` never returns. `sample` on the stuck process shows the main thread parked here:
```
~DynamicModulesIntegrationTest -> ~HttpIntegrationTest -> ~BaseIntegrationTest
-> ~IntegrationTestServerImpl -> IntegrationTestServer::~IntegrationTestServer()
-> Thread::PosixThread::join() -> _pthread_join -> __ulock_wait
```
and the thread it is waiting on parked here:
```
thread_start -> _pthread_start -> _pthread_exit -> _pthread_tsd_cleanup
-> _sigtramp -> ??? (in )
```
So the server thread is exiting, macOS runs its pthread thread-specific-data destructors, one of them enters the loaded Go module, and it never comes back. `pthread_join` then blocks forever.
Repro: any test with a `direct_response` route and a Go module filter, which is what I was adding for #46901. It also reproduces with the in-tree `passthrough` filter, so it has nothing to do with what the filter does — only that a Go module is loaded. `rust`, `rust_static` and `cpp` on the identical test finish in under a second. Go is the only one of the three SDKs whose runtime installs pthread TSD destructors, which fits.
Two things suggest this is macOS-only and test-only rather than a production problem. A Go dynamic module in a long-running Linux Envoy container handles the same `direct_response` route fine over both HTTP/1.1 and HTTP/2 — but that process never tears a server thread down with the module loaded, so it does not exercise this path. And CI is Linux, so this likely never shows up there.
Not sure what the right fix is. Possibly the server thread should not be joined while a dynamic module holding thread-local state is still loaded, or the module should be unloaded first, or this is a Go runtime interaction that Envoy can only work around.
Repro harness: https://github.com/derekargueta/envoy — the test is in the #46902 branch, which skips the `go` param on Apple platforms and cites this issue.
main @ 8ea3b35553, macOS arm64.
Contributor guide
Research direction
Use the linked repro harness and the #46902 test, starting with DynamicModulesIntegrationTest teardown and the IntegrationTestServerImpl to Thread::PosixThread::join stack. Compare the Go dynamic-module case with rust, rust_static, and cpp, then verify that the direct_response test request still succeeds and teardown returns on macOS without hanging.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, go, macos
- Domain
- operating-systems, testing
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100