firecracker-microvm / firecracker-microvm/firecracker

[Feature Request] Host-side guest vmcore dump on kernel panic (pvpanic)

Open
#6,067 2 comments 0 reactions 0 assignees View on GitHub
Status: Awaiting author
Dominant language
Rust
Stars
36.7k
Forks
2.6k
Avg merge
3d 3h
Merged PRs (30d)
58

Description

# Feature Request

Automatically capture a guest `vmcore` on kernel panic via a host-side `pvpanic` device, without reserving any guest memory.

Today the only supported way to obtain a full memory image of a panicked guest kernel is in-guest kdump/kexec, which requires reserving memory via the `crashkernel=` kernel command line parameter — in practice a fixed ~100–200 MB per VM. For microVMs, which are often provisioned with only a few hundred MB of RAM, that reservation consumes a large fraction of guest memory.

But the VMM already has the entire guest physical memory mapped. If the guest signals the host on panic, the host can produce a complete crash dump itself, with zero guest-side memory reservation.

## Describe the desired solution

Add an opt-in, per-VM paravirtualized `pvpanic` device. The guest-side driver is already in mainline Linux (`CONFIG_PVPANIC`), so the guest needs no changes, it simply signals the device on panic.

When Firecracker receives that event, it automatically writes a `vmcore` to a configured host path that is directly analyzable with `crash`/`gdb`, with no in-guest `crashkernel=` reservation.

## Describe possible alternatives

- In-guest kdump via `crashkernel=`. Works, but requires a fixed ~100–200 MB reservation per guest.
- Serial-console panic logs only. No memory reservation, but only yields the backtrace/log text.

## Additional context

I have a working x86_64 PoC (the `pvpanic` device plus the host-side ELF `vmcore` dumper) that produces `crash`/`gdb`-analyzable dumps, with negligible impact on Firecracker's process RSS.

However, I'm not sure whether a full guest vmcore dump really fits Firecracker's scope, given the minimalist core tenet, or whether a lighter mechanism would be preferred. I also noticed that `pvpanic` config isn't enabled in the CI kernels.

If there's a better way to dump a panicked kernel without adding a new device, it would also be much appreciated. And if the community is interested in this feature, I'm very happy to open a PR for this.

## Checks

- [x] Have you searched the Firecracker Issues database for similar requests?
- [x] Have you read all the existing relevant Firecracker documentation?
- [x] Have you read and understood Firecracker's core tenets?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the proposed pvpanic device and the host-side ELF vmcore dumper described in the issue, then check how the CI kernels handle pvpanic configuration. Compare the requested crash/gdb-analyzable output with Firecracker's minimalist core tenets and existing alternatives. Done means an agreed scope and an implementable design, with the relevant guest-panic path and dump behavior validated.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, rust
Domain
operating-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.