Slow vfs for directories with large number of files

### Description

Gvisor (with vfs2) is very slow when accessing directories with a huge number (50000) of files on an external mount. Operations inside such a directory can take hundreds of milliseconds. Even a simple getdents64 syscall can take a very long time as gvisor performs a stat on every file in the directory. Accessing a non-existent file leads to similar behaviour. The performance difference compared to native access is enormous as gvisor is 100x slower in these cases.

### Steps to reproduce

Slow getdents64 performance

1. Create an external mount containing a directory with a huge number (50000) of files.
2. Do a getdents64 syscall from within gvisor
3. See that givsor performs a stat syscall for every file in the directory

A similar issue is that trying to access a non-existent file in the directory

1. Create an external mount containing a directory with a huge number (50000) of files.
2. Try to access a non-existent file in the directory `time cat DOES_NOT_EXIST`
3. See that a simple cat with ENOENT takes hundreds of milliseconds (and fails with a simple ENOENT)

### runsc version

```shell
release-20210906.0
```

### docker version (if using docker)

```shell
20.10.5
```

### uname

5.10.0-8 (debian bullseye)

### kubectl (if using Kubernetes)

_No response_

### repo state (if built from source)

_No response_

### runsc debug logs (if available)

_No response_