firecracker-microvm / firecracker-microvm/nsdi2020-data
QEMU MicroVM: why such bad results and why not in the Firecracker paper?
- Dominant language
- Shell
- Stars
- 26
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
Dear Firecracker paper authors,
While looking at some other paper (https://github.com/rellermeyer/container_benchmarks/issues/1), I noticed the unexplainably bad results of QEMU's MicroVM in your GitHub repo.
Here is the QEMU MicroVM command line: https://github.com/firecracker-microvm/nsdi2020-data/blob/cf1b30f674523ee44363f74e16bcb301ffb10598/scripts/util_start_qemu_uvm.sh
Here is the commit that added QEMU MicroVM data: https://github.com/firecracker-microvm/nsdi2020-data/commit/b35147fbc1e51754757c6aaad04dcd219b1f45df
You also updated all the plots, some of them include QEMU MicroVM perf, especially this one: https://github.com/firecracker-microvm/nsdi2020-data/blob/master/data/m5d.metal/boot-serial-all.pdf
If I'm reading the plot correctly, and if I'm not confusing two shades of "light blue", then QEMU MicroVM exhibits extremely bad boot time (~600ms, about 2.5x worse than normal QEMU).
I also notice that your final paper https://www.usenix.org/system/files/nsdi20-paper-agache.pdf doesn't mention QEMU MicroVM at all (this may be just timing issues, since your GitHub updates are dated with 9. February and the NSDI conference was on 25. February).
So I have two questions:
- Do you have any explanations on QEMU MicroVM boot times? Did you maybe expand on this in some follow-up paper, video, blog, etc?
- How can you explain the fact that QEMU's MicroVM behaves worse than normal QEMU ( with normal machine type)?
- Why didn't you include any mentions of QEMU's MicroVM in the final paper?
Contributor guide
Research direction
Start with scripts/util_start_qemu_uvm.sh, the commit that added QEMU MicroVM data, and data/m5d.metal/boot-serial-all.pdf; compare the command line with the reported boot-time results. Read the linked NSDI paper and benchmark context, then document an evidence-based explanation for the performance difference and the paper's omission in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell
- Domain
- documentation, performance
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100