google / google/gvisor

In-place CPU resize doesn't update the guest CPU count (nproc stays at the boot value)

Open
#14,141 0 comments 0 reactions 0 assignees View on GitHub
area: performance area: platform
Dominant language
Go
Stars
19.3k
Forks
2k
Avg merge
3d 5h
Merged PRs (30d)
264

Description

I hit this using gVisor pods with Kubernetes in-place pod resize. When I raise a pod's CPU
limit at runtime, the host cgroup quota is updated correctly, but the guest keeps reporting the
CPU count it had at boot. Anything that sizes its parallelism off the reported CPU count — Go's
GOMAXPROCS, `make -j$(nproc)`, the JVM's availableProcessors, nginx `worker_processes auto` —
stays capped at the old count and never uses the CPU I just added, unless I recreate the pod.

As far as I can tell this is the guest-visibility half of #12790. That change made `runsc update`
write the new host cgroup values, which works, but nothing tells the running sentry, so the guest
CPU count (ApplicationCores) is still whatever was computed at sandbox boot.

Environment:

runsc release-20260810.0 (spec 1.2.1), systrap platform
kind v0.31.0, Kubernetes v1.35.0 (in-place resize is GA), containerd 2.2.0
node: Linux 6.12.76-linuxkit aarch64

Steps to reproduce. Boot a gVisor pod requesting 1 CPU, check nproc, then resize it to 8 CPUs
in place (no restart) and check nproc again:

apiVersion: v1
kind: Pod
metadata:
name: gv-bug
spec:
runtimeClassName: gvisor
containers:
- name: c
image: busybox
command: ["sleep","3600"]
resizePolicy:
- resourceName: cpu
restartPolicy: NotRequired
resources:
requests: {cpu: "1", memory: "128Mi"}
limits: {cpu: "1", memory: "128Mi"}

What I see:

# booted at cpu=1
$ kubectl exec gv-bug -- nproc
2

# in-place resize to cpu=8
$ kubectl patch pod gv-bug --subresource resize \
--patch '{"spec":{"containers":[{"name":"c","resources":{"requests":{"cpu":"8","memory":"128Mi"},"limits":{"cpu":"8","memory":"128Mi"}}}]}}'
pod/gv-bug patched

# the resize took effect: allocated CPU is 8, no restart, host cgroup quota moved to 8 cores
$ kubectl get pod gv-bug -o jsonpath='{.status.containerStatuses[0].allocatedResources.cpu}'
8
$ kubectl get pod gv-bug -o jsonpath='{.status.containerStatuses[0].restartCount}'
0
# pod cgroup cpu.max, on the node:
800000 100000

# but the guest still reports the boot-time count
$ kubectl exec gv-bug -- nproc
2

The sentry's debug log shows it started with 2 CPUs, and that's the only place the count is ever
set — nothing recomputes it after the resize:

I0816 16:27:55.856813 1 loader.go:580] CPUs: 2

Expected: after an in-place CPU resize the guest should see the updated CPU count. If updating a
running sentry's CPU count isn't feasible, a supported way to boot the sandbox with a fixed CPU
count independent of the quota would also solve it — you could boot with headroom and let the
in-place quota change (which already works) govern the real CPU, without a restart.

Actual: the guest CPU count is frozen at the boot value, so the added CPU is unusable for any
workload that sizes itself to nproc until the pod is recreated.

Contributor guide

Open the contributing guide

Research direction

Start at the sentry CPU initialization reported at loader.go:580 and trace how ApplicationCores is computed at sandbox boot. Then inspect the runsc update path and how it could notify a running sentry; done means an in-place CPU resize changes the guest count observed by nproc without restarting the pod.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes, linux
Domain
devops, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.