System-wide kernel freeze when loading models via GPUStack worker (Python processes stuck in D-state at rwsem_down_read_slowpath, no Xid / no OOM)

GPUStack version

v2.0.1

Operating System & CPU Architecture

ubuntu22.04;ubuntu24.04

GPU

NVIDIA RTX 4090

▶️ Steps to reproduce

Environment (Current)

• GPU: NVIDIA RTX 4090 × 8
• CPU: Intel Xeon Platinum 6442Y (192 cores)
• Memory: 1 TB
• OS: Ubuntu 24.04 LTS
• Kernel: 6.8.0-90-generic
• NVIDIA Driver: 580.95.05
• Docker: Docker 29.1.3
• Model: Qwen3-VL-32B-Instruct

GPUStack

• Server image: gpustack/gpustack:latest
• Worker image: gpustack/gpustack:latest
  same as v2.0.1

Summary

When starting models via GPUStack worker, the host system may enter a kernel-level deadlock, resulting in a full machine freeze.

Once triggered:

• Multiple Python worker processes enter uninterruptible sleep (D-state)
• All affected processes become unkillable
• nvidia-smi executed on the host blocks indefinitely
• No NVIDIA Xid errors, OOM events, or GPU resets are logged
• The system cannot recover without a forced reboot

This issue has reproduced consistently and only occurs when models are launched via GPUStack.

---

Expected Behavior

• Model initialization should not cause kernel deadlock

• nvidia-smi should remain responsive

• In failure scenarios, expected outcomes would be:

  • CUDA OOM
  • vLLM error
  • container crash

A full system freeze is not expected.

---

Actual Behavior

• Host becomes unresponsive during GPUStack model startup
• Multiple Python processes enter state:D
• Kernel reports hung tasks
• No recovery is possible without reboot

---

Docker Deployment Configuration

The issue occurs when GPUStack is deployed using the following Docker commands.

GPUStack Server

sudo docker run -d \
  --name gpustack \
  --restart unless-stopped \
  -p 9090:80 \
  -v gpustack-data:/var/lib/gpustack \
  gpustack/gpustack:latest

---

GPUStack Worker

sudo docker run -d \
  --name gpustack-worker \
  --restart unless-stopped \
  --privileged \
  --network=host \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  --volume gpustack-data:/var/lib/gpustack \
  --volume /dataSSD:/dataSSD \
  --runtime nvidia \
  gpustack/gpustack:latest \
  --server-url http://<WORKER_IP>:9090 \
  --token <REDACTED> \
  --advertise-address <WORKER_IP>

---

Kernel Hung Task Evidence (Captured During Freeze)

During the freeze, multiple Python worker processes are reported by the kernel as blocked for extended periods.
All affected processes share an identical call trace.

INFO: task python blocked for more than 30 seconds.
task:python state:D
Call Trace:
  __schedule
  schedule
  schedule_preempt_disabled
  rwsem_down_read_slowpath
  down_read
  do_user_addr_fault
  exc_page_fault
  asm_exc_page_fault

Multiple Python processes exhibit the same stack trace, indicating a systemic kernel lock contention or deadlock, not a single-thread stall.

---

Technical Interpretation (Key Observation)

• All affected processes are Python worker processes
• All are stuck in uninterruptible sleep (D-state)
• All block in rwsem_down_read_slowpath while handling user-space page faults
• No GPU Xid, OOM, or reset events are recorded
• Once triggered, nvidia-smi also blocks

This strongly suggests a kernel-level deadlock, likely involving:

• NVIDIA driver
• UVM (Unified Virtual Memory)
• GPU memory mapping or page fault handling under multi-process conditions

---

Comparison Test (Important)

On the same server, using:

• Official vLLM Docker image: vllm/vllm-openai:v0.11.0-x86_64
• Same models
• Same NVIDIA driver (580.95.05)
• Same OS (Ubuntu 24.04)

👉 No system freeze occurs, even when running nvidia-smi during model loading.

This strongly suggests the issue is specific to GPUStack’s worker orchestration or GPU interaction path, rather than hardware, OS, or vLLM itself.

---

Cross-Version Reproducibility

This issue was originally observed on Ubuntu 22.04 LTS and remains reproducible after upgrading the system to Ubuntu 24.04 LTS for verification purposes.

• Ubuntu 22.04 LTS

  • Kernel 5.15.x
  • NVIDIA Driver 565.77

• Ubuntu 24.04 LTS

  • Kernel 6.8.0-90-generic
  • NVIDIA Driver 580.95.05

In both environments:

• The system freeze occurs only when models are launched via GPUStack worker
• Python worker processes enter D-state at rwsem_down_read_slowpath
• No NVIDIA Xid, OOM, or GPU reset events are logged
• Official vLLM Docker images do not exhibit this behavior

This indicates the issue is not tied to a specific OS, kernel, or NVIDIA driver version.

---

Questions / Request for Guidance

1. Does GPUStack worker perform any GPU probing, memory inspection, or synchronization that could interact with NVIDIA UVM or page fault handling?

2. Are there known issues related to:

   • Multi-process GPU access
   • Worker restarts
   • Interaction with nvidia-smi

3. Are there recommended debug flags, environment variables, or runtime options to help further isolate this behavior?
   Any guidance would be greatly appreciated.

❌ Actual result

Kernel Hung Task Evidence

The following kernel logs were captured during the system freeze.
Multiple Python worker processes entered uninterruptible sleep (D-state), all blocked in the same kernel path.

Dec 15 11:06:35 node1 kernel: INFO: task python:20948 blocked for more than 30 seconds.
Dec 15 11:06:35 node1 kernel: task:python state:D pid:20948 ppid:19373
Dec 15 11:06:35 node1 kernel: Call Trace:
  __schedule
  schedule
  schedule_preempt_disabled
  rwsem_down_read_slowpath
  down_read
  do_user_addr_fault
  exc_page_fault
  asm_exc_page_fault

Dec 15 11:07:07 node1 kernel: INFO: task python:19649 blocked for more than 30 seconds.
Dec 15 11:07:07 node1 kernel: task:python state:D pid:19649 ppid:19373
Dec 15 11:07:07 node1 kernel: Call Trace:
  __schedule
  schedule
  schedule_preempt_disabled
  rwsem_down_read_slowpath
  down_read
  do_user_addr_fault
  exc_page_fault
  asm_exc_page_fault

Dec 15 11:07:07 node1 kernel: INFO: task python:19655 blocked for more than 30 seconds.
Dec 15 11:07:07 node1 kernel: task:python state:D pid:19655 ppid:19373
Dec 15 11:07:07 node1 kernel: Call Trace:
  __schedule
  schedule
  schedule_preempt_disabled
  rwsem_down_read_slowpath
  down_read
  do_user_addr_fault
  exc_page_fault
  asm_exc_page_fault

---