oxidecomputer / oxidecomputer/propolis
server: VmmHdl appears to be leaked during normal VM shutdown
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 270
- Forks
- 42
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 6
Description
Repro steps:
- Start an ad hoc propolis-server instance
- Send it an ensure request
- Ask to stop the VM instance (you don't have to start it first, though you can)
Observed: The DESTROY_SELF vmm ioctl is issued (and a probe set on vmm_destroy_locked fires), but the kernel VMM persists until the process is killed. The stack on the resulting call to vmm_destroy_finish shows it originated from genunix!proc_exit. Writing a simple Drop impl for VmmHdl that just prints to stderr shows that this drop impl is apparently never reached.
Expected: there is at least some way to convince Propolis to fully close the kernel VMM fd on VM destruction.
We've discussed this in the past and concluded that in at least some cases it's useful for the kernel VMM to outlive the Propolis instance that owns it so that the VMM can be inspected with tools like mdb -b. I would at least like to consider avoiding this for production builds, though, for reasons related to this Omicron issue comment: it's useful for sled-agent to be able to say "Propolis reported that it's in the Destroyed state, so all its reservoir memory is released," because this helps give it the ability to tell Nexus that a VMM is gone and then do long-running zone cleanup operations afterward.
Even absent that motivation, I'd at least like to understand exactly which paths aren't fully dropping their VmmHdl references so that we can adjust the behavior if/when we need to.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the ad hoc propolis-server ensure-and-stop flow and trace the DESTROY_SELF path through vmm_destroy_locked and vmm_destroy_finish. Inspect VmmHdl ownership and Drop behavior across VM destruction, including the paths described in the issue. Done means the relevant references are understood and a production-appropriate way to close the kernel VMM is identified, while preserving any intended inspection behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100