boot: nil-pointer panic in startSubcontainer kills the sandbox when joining a PID namespace whose container never started
- Dominant language
- Go
- Stars
- 19.3k
- Forks
- 2k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 275
Description
## Summary
`Loader.startSubcontainer` dereferences `p.tg` without a nil check when a container joins another
container's PID namespace. If the container that owns that namespace was registered but its
process was never created, `p.tg` is nil and the **sentry panics** — which kills the entire
sandbox, not just the container being started.
We hit this twice on one node, on different pods and different days, with an identical panic PC.
## The panic
```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x102a411]
gvisor.dev/gvisor/pkg/sentry/kernel.(*ThreadGroup).PIDNamespace(...)
pkg/sentry/kernel/threads.go:499
gvisor.dev/gvisor/runsc/boot.(*Loader).startSubcontainer(...)
runsc/boot/loader.go:1219 +0x12f1
gvisor.dev/gvisor/runsc/boot.(*containerManager).StartSubcontainer(...)
runsc/boot/controller.go:495 +0xf28
...
gvisor.dev/gvisor/pkg/urpc.(*Server).handleOne(...)
```
Because the panic happens inside the urpc handler goroutine, it takes down the **sentry**. Every
container in that pod dies, not just the one being started.
## The bug
`runsc/boot/loader.go`, `startSubcontainer`:
```go
for _, p := range l.processes {
if ns.Path == p.pidnsPath {
log.Debugf("Joining PID namespace named %q", ns.Path)
pidns = p.tg.PIDNamespace() // <-- line 1303: no nil check on p.tg
break
}
}
if pidns == nil { // <-- the "not found" case IS handled
log.Warningf("PID namespace %q not found, running in new PID namespace", ns.Path)
pidns = l.k.RootPIDNamespace().NewChild(...)
}
```
The `pidns == nil` fallback shows the author anticipated *"no container owns this namespace"*. What
isn't handled is *"a container owns it, but has no process"*.
That state is easy to reach, because a container advertises its namespace path **long before** its
`ThreadGroup` exists — in the same function:
```
line 1312 ep.pidnsPath = ns.Path <-- advertised here
line 1335 return fmt.Errorf("creating new process: %w", err)
line 1341 return fmt.Errorf("using TTY, stdios not expected: %d", l)
line 1344 return fmt.Errorf("terminal enabled but no TTY provided...")
line 1361 ep.tg, ep.tty, err = l.createContainerProcess(info) <-- tg set here
line 1363 return err
```
Fail on **any** of those four returns and the container stays in `l.processes` with `pidnsPath`
set and `tg` still nil. The next container to join that path finds it and panics the sandbox.
```mermaid
flowchart TD
A["container A: startSubcontainer"] --> B["line 1312
ep.pidnsPath = ns.Path"]
B --> C{"fails before line 1361?"}
C -->|"yes — any of 4 returns"| D["A stays in l.processes
pidnsPath SET · tg NIL"]
C -->|no| E["A.tg assigned · healthy"]
D --> F["container B: startSubcontainer
joins the same ns.Path"]
F --> G["loop matches A
pidns = A.tg.PIDNamespace()"]
G --> H["NIL DEREF · SENTRY PANICS
whole sandbox dies"]
style H fill:#c0392b,color:#fff,stroke:#900
```
## Steps to reproduce
This needs no race — the failure is deterministic. The `terminal` check at line 1344 is the
easiest trigger.
1. Start a sandbox (root container) as usual.
2. Build a bundle for **container A** whose spec has:
- a PID namespace entry with a non-empty `path`, e.g.
`"linux": {"namespaces": [{"type": "pid", "path": "/proc//ns/pid"}]}`
- `"process": {"terminal": true, ...}`
3. `runsc create --bundle= cidA` — **without** `--console-socket`.
4. `runsc start cidA` → fails with
`terminal enabled but no TTY provided. Did you set --console-socket on create?`
Container A is now registered with `pidnsPath` set and `tg == nil`.
5. Build a bundle for **container B** with the *same* PID namespace `path` and
`"terminal": false`.
6. `runsc create --bundle= cidB && runsc start cidB`
Step 6 panics the sentry and destroys the sandbox. Expected instead: B either joins a usable
namespace or falls back to a new one (the code three lines below already does exactly that).
Any of the other three error paths reproduces it equally well.
## How we hit it in production (Kubernetes)
Pods with **`shareProcessNamespace: true`** — every container joins the same PID namespace path,
so a single container failing to start turns the *next* container's start into a sandbox-wide
panic. Our pods are a workload container plus a sidecar; when the workload container's start
failed, starting the sidecar killed the whole pod.
## Impact
- One container failing to start destroys **every** container in the pod.
- Worse downstream: the dead sandbox leaves `containerd-shim-runsc-v1` processes that can never be
reaped, and a later containerd restart hangs on them permanently. On our node that produced a
4-day containerd crash loop starting ~19 hours after the original pod failure, with nothing
obvious linking the two events. Details in a companion issue we're filing about
`Init.delete()`; see also containerd/containerd#13848.
## Environment
- gVisor `release-20260615.0-66-g0f3a32472c9d`, `release-20260622.0-1-g2f05ec978e4c`.
- `platform=systrap`, `overlay2=all:self`
- containerd 2.3.1, Kubernetes 1.36.1 , `RuntimeClass: runsc`
- Observed twice on one node: 2026-07-30 and 2026-08-04, different pods, identical panic PC
(`0x102a411`).
- uname : Linux worker-9 6.8.0-136-generic #136-Ubuntu SMP PREEMPT_DYNAMIC Wed Jul 1 21:53:05 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Happy to open a PR.
Contributor guide
Research direction
Start in runsc/boot/loader.go at Loader.startSubcontainer and trace the l.processes lookup through the provided reproduction steps, especially the terminal error path. Verify the behavior when a matching process has no ThreadGroup, then run the relevant runsc boot tests or reproduction and confirm that starting the second container no longer panics the sentry.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes, linux
- Domain
- devops, operating-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100