firecracker-microvm / firecracker-microvm/firecracker
[Devices] Offer support for hardware-accelerated inference in Firecracker
Open
Status: Parked
Type: Enhancement
- Dominant language
- Rust
- Stars
- 36.7k
- Forks
- 2.6k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 53
Description
Doing hardware-accelerated inference in a serverless environment is compelling use case.
However, adding straight up GPU passthrough means that microVM can't oversubscribe memory, and we need to add PCI emulation to Firecracker, which comes with a lot of extra complexity/attack surface.
The first step here will be to research the options and alternatives (e.g., GPU passthrough, or something else), and figure out the path forward.
Related issues: #849, #776.
Contributor guide
Assessment
This issue has not been assessed yet.