facebook / facebook/hhvm

memfd_init() can't handle unlimited size tmpfs

Open
#8,470 5 comments 0 reactions 0 assignees View on GitHub
hack
Dominant language
C++
Stars
18.7k
Forks
3.1k
Avg merge
1h 47m
Merged PRs (30d)
2

Description

### HHVM Version

> Please include the output of `hhvm --version` and `hh_client --version`

I've actually reproduced this issue with flow 0.95.2, which includes [a fork](https://github.com/facebook/flow/blob/v0.95.1/hack/heap/hh_shared.c#L672) of hack code, but it should affect any version of hhvm since https://github.com/facebook/hhvm/commit/a8fdccafc59768f2ff8883ff9d236e7ac322b21f.

### Operating System and Version

> For example, 'Debian Squeeze', 'Ubuntu 16.04', 'MacOS High Sierra'. Please
> also mention if you are using Docker or similar images.

Any OS with a Linux kernel missing `memfd_create`. i.e., Linux <3.17.

### Standalone code, or other way to reproduce the problem

> This should not depend on installing any libraries or frameworks. Ideally, it
> should be possible to copy-paste this into a single file and reproduce the
> problem by running `hhvm` and/or `hh_client`

This issue can be reproduced on any Linux OS with a kernel older than 3.17. Here's what I did to reproduce the issue by downgrading the kernel on an Ubuntu 14.04 VM (definitely a hacky way to get an old kernel):

```shell
$ uname -a
Linux ubuntu-trusty-1 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ sudo apt-get update
$ sudo apt-get install linux-generic-lts-trusty unzip
$ sudo apt-get purge linux-image-4.4.0-143-generic # don't abort removal
# Hack! Make sure the kernel is really really gone.
$ sudo rm /boot/vmlinuz-4.4.0-143-generic /boot/initrd.img-4.4.0-143-generic
$ sudo update-grub
$ sudo reboot
...
$ uname -a
Linux ubuntu-trusty-1 3.13.0-167-generic #217-Ubuntu SMP Wed Mar 13 16:18:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ sudo mount -t tmpfs tmpfs -o size=0 /dev/shm
$ sudo mount -t tmpfs tmpfs -o size=0 /tmp
$ wget https://github.com/facebook/flow/releases/download/v0.95.2/flow-linux64-v0.95.2.zip
$ unzip flow-linux64-v0.95.2.zip
$ flow/flow init
$ flow/flow check
Out of shared memory:
Raised by primitive operation at file "hack/heap/sharedMem.ml", line 99, characters 6-89
```

### Actual result

> If applicable, please copy-paste output from `hhvm` or `hh_client`

```shell
$ flow/flow check
Out of shared memory:
Raised by primitive operation at file "hack/heap/sharedMem.ml", line 99, characters 6-89
```

The failing logic is in [memfd_init](https://github.com/facebook/hhvm/blob/master/hphp/hack/src/heap/hh_shared.c#L672), and can be seen with strace:

```
syscall_319(0xb27a60, 0, 0x680a00000, 0x1, 0x680800000, 0x7fd7ffa4f0c0) = -1 (errno 38)
stat("/run/shm", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=40, ...}) = 0
statfs("/run/shm", {f_type=0x1021994, f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=473771, f_ffree=473770, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
stat("/run/shm", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=40, ...}) = 0
stat("/tmp/flow", {st_mode=S_IFDIR|0777, st_size=80, ...}) = 0
statfs("/tmp/flow", {f_type=0x1021994, f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=473771, f_ffree=473743, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
stat("/tmp/flow", {st_mode=S_IFDIR|0777, st_size=80, ...}) = 0
write(2, "Out of shared memory:\nRaised by "..., 112Out of shared memory:
Raised by primitive operation at file "hack/heap/sharedMem.ml", line 99, characters 6-89

) = 112
exit_group(15) = ?
```

This kernel doesn't support memfd_create, so it falls back to creating a temporary file in /run/shm or /tmp/flow. First, [assert_avail_exceeds_minimum](https://github.com/facebook/hhvm/blob/master/hphp/hack/src/heap/hh_shared.c#L644) tries to assert that the filesystem has more than some minimum threshold of space. It computes the space as `f_bsize * fbavail`. From strace, we can see that `f_bavail` is zero.

This is a [special case](https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt) for tmpfs: "If nr_blocks=0 (or size=0), blocks will not be limited in that instance". i.e., if `f_bavail == 0` on tmpfs, the mount has unlimited size. `assert_avail_exceeds_minimum` does not understand this case and decides there is not enough space.

### Expected result

Flow works.

`assert_avail_exceeds_minimum` should assume that an unlimited tmpfs has enough space.

### Note

This was discovered while investigating a [report that flow does not work in gVisor](https://github.com/google/gvisor/issues/146). gVisor does not implement `memfd_create` and creates unlimited tmpfs mounts by default, thus triggering this bug. That said, it will support `memfd_create` very soon, which will mask this bug and allow things to work.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.