firecracker-microvm / firecracker-microvm/firecracker
[Feature Request] Host-side guest vmcore dump on kernel panic (pvpanic)
- Dominant language
- Rust
- Stars
- 36.7k
- Forks
- 2.6k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 58
Description
# Feature Request
Automatically capture a guest `vmcore` on kernel panic via a host-side `pvpanic` device, without reserving any guest memory.
Today the only supported way to obtain a full memory image of a panicked guest kernel is in-guest kdump/kexec, which requires reserving memory via the `crashkernel=` kernel command line parameter — in practice a fixed ~100–200 MB per VM. For microVMs, which are often provisioned with only a few hundred MB of RAM, that reservation consumes a large fraction of guest memory.
But the VMM already has the entire guest physical memory mapped. If the guest signals the host on panic, the host can produce a complete crash dump itself, with zero guest-side memory reservation.
## Describe the desired solution
Add an opt-in, per-VM paravirtualized `pvpanic` device. The guest-side driver is already in mainline Linux (`CONFIG_PVPANIC`), so the guest needs no changes, it simply signals the device on panic.
When Firecracker receives that event, it automatically writes a `vmcore` to a configured host path that is directly analyzable with `crash`/`gdb`, with no in-guest `crashkernel=` reservation.
## Describe possible alternatives
- In-guest kdump via `crashkernel=`. Works, but requires a fixed ~100–200 MB reservation per guest.
- Serial-console panic logs only. No memory reservation, but only yields the backtrace/log text.
## Additional context
I have a working x86_64 PoC (the `pvpanic` device plus the host-side ELF `vmcore` dumper) that produces `crash`/`gdb`-analyzable dumps, with negligible impact on Firecracker's process RSS.
However, I'm not sure whether a full guest vmcore dump really fits Firecracker's scope, given the minimalist core tenet, or whether a lighter mechanism would be preferred. I also noticed that `pvpanic` config isn't enabled in the CI kernels.
If there's a better way to dump a panicked kernel without adding a new device, it would also be much appreciated. And if the community is interested in this feature, I'm very happy to open a PR for this.
## Checks
- [x] Have you searched the Firecracker Issues database for similar requests?
- [x] Have you read all the existing relevant Firecracker documentation?
- [x] Have you read and understood Firecracker's core tenets?
Contributor guide
Research direction
Start by reviewing the proposed pvpanic device and the host-side ELF vmcore dumper described in the issue, then check how the CI kernels handle pvpanic configuration. Compare the requested crash/gdb-analyzable output with Firecracker's minimalist core tenets and existing alternatives. Done means an agreed scope and an implementable design, with the relevant guest-panic path and dump behavior validated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, rust
- Domain
- operating-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100