memfd_init() can't handle unlimited size tmpfs
- Dominant language
- C++
- Stars
- 18.7k
- Forks
- 3.1k
- Avg merge
- 1h 47m
- Merged PRs (30d)
- 2
Description
### HHVM Version
> Please include the output of `hhvm --version` and `hh_client --version`
I've actually reproduced this issue with flow 0.95.2, which includes [a fork](https://github.com/facebook/flow/blob/v0.95.1/hack/heap/hh_shared.c#L672) of hack code, but it should affect any version of hhvm since https://github.com/facebook/hhvm/commit/a8fdccafc59768f2ff8883ff9d236e7ac322b21f.
### Operating System and Version
> For example, 'Debian Squeeze', 'Ubuntu 16.04', 'MacOS High Sierra'. Please
> also mention if you are using Docker or similar images.
Any OS with a Linux kernel missing `memfd_create`. i.e., Linux <3.17.
### Standalone code, or other way to reproduce the problem
> This should not depend on installing any libraries or frameworks. Ideally, it
> should be possible to copy-paste this into a single file and reproduce the
> problem by running `hhvm` and/or `hh_client`
This issue can be reproduced on any Linux OS with a kernel older than 3.17. Here's what I did to reproduce the issue by downgrading the kernel on an Ubuntu 14.04 VM (definitely a hacky way to get an old kernel):
```shell
$ uname -a
Linux ubuntu-trusty-1 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ sudo apt-get update
$ sudo apt-get install linux-generic-lts-trusty unzip
$ sudo apt-get purge linux-image-4.4.0-143-generic # don't abort removal
# Hack! Make sure the kernel is really really gone.
$ sudo rm /boot/vmlinuz-4.4.0-143-generic /boot/initrd.img-4.4.0-143-generic
$ sudo update-grub
$ sudo reboot
...
$ uname -a
Linux ubuntu-trusty-1 3.13.0-167-generic #217-Ubuntu SMP Wed Mar 13 16:18:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ sudo mount -t tmpfs tmpfs -o size=0 /dev/shm
$ sudo mount -t tmpfs tmpfs -o size=0 /tmp
$ wget https://github.com/facebook/flow/releases/download/v0.95.2/flow-linux64-v0.95.2.zip
$ unzip flow-linux64-v0.95.2.zip
$ flow/flow init
$ flow/flow check
Out of shared memory:
Raised by primitive operation at file "hack/heap/sharedMem.ml", line 99, characters 6-89
```
### Actual result
> If applicable, please copy-paste output from `hhvm` or `hh_client`
```shell
$ flow/flow check
Out of shared memory:
Raised by primitive operation at file "hack/heap/sharedMem.ml", line 99, characters 6-89
```
The failing logic is in [memfd_init](https://github.com/facebook/hhvm/blob/master/hphp/hack/src/heap/hh_shared.c#L672), and can be seen with strace:
```
syscall_319(0xb27a60, 0, 0x680a00000, 0x1, 0x680800000, 0x7fd7ffa4f0c0) = -1 (errno 38)
stat("/run/shm", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=40, ...}) = 0
statfs("/run/shm", {f_type=0x1021994, f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=473771, f_ffree=473770, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
stat("/run/shm", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=40, ...}) = 0
stat("/tmp/flow", {st_mode=S_IFDIR|0777, st_size=80, ...}) = 0
statfs("/tmp/flow", {f_type=0x1021994, f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=473771, f_ffree=473743, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
stat("/tmp/flow", {st_mode=S_IFDIR|0777, st_size=80, ...}) = 0
write(2, "Out of shared memory:\nRaised by "..., 112Out of shared memory:
Raised by primitive operation at file "hack/heap/sharedMem.ml", line 99, characters 6-89
) = 112
exit_group(15) = ?
```
This kernel doesn't support memfd_create, so it falls back to creating a temporary file in /run/shm or /tmp/flow. First, [assert_avail_exceeds_minimum](https://github.com/facebook/hhvm/blob/master/hphp/hack/src/heap/hh_shared.c#L644) tries to assert that the filesystem has more than some minimum threshold of space. It computes the space as `f_bsize * fbavail`. From strace, we can see that `f_bavail` is zero.
This is a [special case](https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt) for tmpfs: "If nr_blocks=0 (or size=0), blocks will not be limited in that instance". i.e., if `f_bavail == 0` on tmpfs, the mount has unlimited size. `assert_avail_exceeds_minimum` does not understand this case and decides there is not enough space.
### Expected result
Flow works.
`assert_avail_exceeds_minimum` should assume that an unlimited tmpfs has enough space.
### Note
This was discovered while investigating a [report that flow does not work in gVisor](https://github.com/google/gvisor/issues/146). gVisor does not implement `memfd_create` and creates unlimited tmpfs mounts by default, thus triggering this bug. That said, it will support `memfd_create` very soon, which will mask this bug and allow things to work.
Contributor guide
Assessment
This issue has not been assessed yet.