firecracker-microvm / firecracker-microvm/firecracker

[Feature Request] Support hugetlbfs memory files for snapshot creation and file-backed restore

Open
#6,099 3 comments 0 reactions 1 assignee Claimed by @marco-marangoni View on GitHub
Status: WIP Type: Enhancement
Dominant language
Rust
Stars
36.7k
Forks
2.6k
Avg merge
3d 2h
Merged PRs (30d)
53

Description

# Feature Request

Allow Firecracker to create snapshot memory files on a 2 MiB hugetlbfs mount and restore those files directly using the `File` memory backend.

Currently, Firecracker supports explicit 2 MiB hugetlbfs pages for newly created microVMs. It can also restore a snapshot into hugetlbfs-backed anonymous memory through UFFD.

However:

- The `File` snapshot backend explicitly rejects `huge_pages: "2M"`.
- The snapshot writer cannot populate a memory file located on hugetlbfs using its current ordinary file-I/O path.
- As a result, explicit hugetlbfs snapshot restore always requires a separate UFFD handler.

UFFD is useful for demand paging, but it requires a separate userspace process and fault-handling protocol. For fully resident snapshots reused by one or more microVMs, a hugetlbfs memory file could instead be mapped directly.

## Describe the desired solution

### Snapshot creation

When `mem_file_path` for `PUT /snapshot/create` refers to a file on a compatible hugetlbfs mount, Firecracker would:

1. Size the file and map it writable with `MAP_SHARED`.
2. Write the full or differential memory snapshot through that mapping.

### Snapshot restore

When `PUT /snapshot/load` uses a `File` backend with `huge_pages: "2M"`, Firecracker would map a compatible hugetlbfs memory file read-write with `MAP_PRIVATE`.

This allows guest writes through copy-on-write without modifying the snapshot file. Existing behavior for regular files and UFFD would remain unchanged.

## Describe possible alternatives

The current workaround is to restore through UFFD, which requires running and managing a separate page-fault handler.

Transparent huge pages can be used with file-backed restore, but they are opportunistic and do not provide deterministic backing from the explicit hugetlbfs pool.

## Additional context

## Checks

- [X] Have you searched the Firecracker Issues database for similar requests?
- [X] Have you read all the existing relevant Firecracker documentation?
- [X] Have you read and understood Firecracker's core tenets?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.